Jul 30 20:55:18.061198 ip-10-0-129-162 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 30 20:55:18.061207 ip-10-0-129-162 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 30 20:55:18.061215 ip-10-0-129-162 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 30 20:55:18.061486 ip-10-0-129-162 systemd[1]: Failed to start Kubernetes Kubelet. Jul 30 20:55:28.053254 ip-10-0-129-162 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 30 20:55:28.053266 ip-10-0-129-162 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 64d2f4ef4cc7474c82016f671454567a -- Jul 30 20:57:09.188856 ip-10-0-129-162 systemd[1]: Starting Kubernetes Kubelet... Jul 30 20:57:09.574279 ip-10-0-129-162 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 20:57:09.574279 ip-10-0-129-162 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 20:57:09.574279 ip-10-0-129-162 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 20:57:09.574279 ip-10-0-129-162 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 20:57:09.574279 ip-10-0-129-162 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 20:57:09.575015 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.574357 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 20:57:09.578757 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578740 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.578757 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578756 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.578757 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578760 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578764 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578767 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578770 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578773 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578775 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578778 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578781 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578784 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578787 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578792 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578795 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578811 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578815 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578818 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578821 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578824 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578827 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578829 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.578849 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578832 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578834 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578837 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578840 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578842 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578846 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578849 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578852 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578855 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578857 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578860 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578863 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578865 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578868 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578870 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578873 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578875 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578878 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578881 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578883 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.579386 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578886 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578889 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578891 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578894 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578896 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578900 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578911 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578914 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578917 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578919 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578922 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578925 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578928 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578930 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578933 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578936 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578938 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578941 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578944 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578946 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.579922 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578949 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578952 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578954 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578957 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578960 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578962 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578964 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578967 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578969 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578972 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578974 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578977 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578979 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578982 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578985 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578987 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578989 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578992 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.578995 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579003 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.580592 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579006 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579009 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579011 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579014 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579016 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579019 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579023 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579026 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579029 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579032 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579034 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579038 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579040 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579043 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579045 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579048 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579051 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.579053 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581216 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.581217 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581228 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581234 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581237 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581240 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581243 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581246 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581248 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581251 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581254 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581256 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581259 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581261 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581264 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581267 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581269 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581272 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581274 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581277 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581280 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581283 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.581721 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581286 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581288 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581291 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581293 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581296 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581298 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581301 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581304 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581306 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581309 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581311 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581313 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581316 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581319 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581322 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581325 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581327 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581330 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581332 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581335 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.582265 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581337 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581340 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581342 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581345 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581347 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581350 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581352 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581355 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581358 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581366 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581369 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581372 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581374 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581376 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581379 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581382 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581384 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581386 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581389 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581392 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.582788 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581394 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581399 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581403 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581406 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581409 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581413 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581416 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581418 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581422 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581425 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581428 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581431 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581433 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581436 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581439 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581443 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581447 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581451 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581453 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.583290 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581456 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581458 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581461 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581464 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581467 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581470 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581472 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581479 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581482 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581485 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581487 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581490 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581492 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581495 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581498 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581500 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581503 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581506 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581509 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.583776 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581930 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581936 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581939 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581943 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581945 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581948 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581950 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581953 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581956 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581959 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581963 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581967 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581970 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581973 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581976 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581979 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581983 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581986 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581989 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.584249 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581992 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581995 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.581999 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582002 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582005 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582008 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582012 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582016 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582019 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582021 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582024 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582027 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582030 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582033 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582036 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582038 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582041 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582044 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582046 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.584737 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582049 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582052 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582054 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582057 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582059 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582061 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582064 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582067 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582069 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582072 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582074 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582078 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582081 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582083 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582085 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582088 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582091 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582093 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582095 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582098 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.585191 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582100 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582102 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582105 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582108 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582110 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582113 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582117 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582119 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582122 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582125 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582127 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582129 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582132 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582134 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582137 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582140 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582142 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582144 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582147 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582150 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.585691 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582152 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582155 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582157 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582160 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582163 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582165 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582168 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582170 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582173 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582175 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582178 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582180 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582183 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582185 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582188 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582190 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582193 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582196 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582199 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582201 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.586218 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582204 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582277 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582291 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582299 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582306 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582313 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582316 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582321 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582326 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582329 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582331 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582335 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582338 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582342 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582345 2567 flags.go:64] FLAG: --cgroup-root="" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582348 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582351 2567 flags.go:64] FLAG: --client-ca-file="" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582354 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582357 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582363 2567 flags.go:64] FLAG: --cluster-domain="" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582366 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582369 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582372 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582376 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 20:57:09.586729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582380 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582382 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582385 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582389 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582392 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582394 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582398 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582401 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582404 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582408 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582411 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582413 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582416 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582419 2567 flags.go:64] FLAG: --enable-server="true" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582422 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582426 2567 flags.go:64] FLAG: --event-burst="100" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582429 2567 flags.go:64] FLAG: --event-qps="50" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582432 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582435 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582438 2567 flags.go:64] FLAG: --eviction-hard="" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582442 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582445 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582448 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582453 2567 flags.go:64] FLAG: --eviction-soft="" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582456 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 20:57:09.587310 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582459 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582462 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582465 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582468 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582471 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582474 2567 flags.go:64] FLAG: --feature-gates="" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582477 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582481 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582483 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582486 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582489 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582492 2567 flags.go:64] FLAG: --help="false" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582495 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582498 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582501 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582504 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582507 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582511 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582514 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582517 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582520 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582523 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582525 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582528 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 20:57:09.587981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582531 2567 flags.go:64] FLAG: --kube-reserved="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582534 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582537 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582540 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582542 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582545 2567 flags.go:64] FLAG: --lock-file="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582549 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582552 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582555 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582560 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582564 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582566 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582569 2567 flags.go:64] FLAG: --logging-format="text" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582572 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582575 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582595 2567 flags.go:64] FLAG: --manifest-url="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582599 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582603 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582607 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582611 2567 flags.go:64] FLAG: --max-pods="110" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582614 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582617 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582620 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582623 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582626 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 20:57:09.588594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582629 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582632 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582639 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582642 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582646 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582649 2567 flags.go:64] FLAG: --pod-cidr="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582652 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582658 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582661 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582664 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582667 2567 flags.go:64] FLAG: --port="10250" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582670 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582673 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0683ce3585c81677f" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582676 2567 flags.go:64] FLAG: --qos-reserved="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582680 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582683 2567 flags.go:64] FLAG: --register-node="true" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582686 2567 flags.go:64] FLAG: --register-with-taints="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582690 2567 flags.go:64] FLAG: --registry-burst="10" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582694 2567 flags.go:64] FLAG: --registry-qps="5" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582697 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582700 2567 flags.go:64] FLAG: --reserved-memory="" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582703 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582706 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582709 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582712 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 20:57:09.589239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582715 2567 flags.go:64] FLAG: --runonce="false" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582718 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582720 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582723 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582726 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582729 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582732 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582735 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582738 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582741 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582744 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582746 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582749 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582752 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582755 2567 flags.go:64] FLAG: --system-cgroups="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582757 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582763 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582765 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582768 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582773 2567 flags.go:64] FLAG: --tls-min-version="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582775 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582780 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582783 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582786 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582789 2567 flags.go:64] FLAG: --v="2" Jul 30 20:57:09.589878 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582793 2567 flags.go:64] FLAG: --version="false" Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582799 2567 flags.go:64] FLAG: --vmodule="" Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582803 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.582806 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582927 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582930 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582934 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582937 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582941 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582943 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582947 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582951 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582954 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582957 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582959 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582962 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582964 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582967 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582969 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582974 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.590481 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582977 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582979 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582982 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582985 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582988 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582991 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582993 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.582996 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583000 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583002 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583005 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583007 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583010 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583013 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583016 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583019 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583021 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583024 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583026 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.590987 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583029 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583031 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583034 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583036 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583039 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583041 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583044 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583046 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583049 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583052 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583054 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583057 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583061 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583063 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583067 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583077 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583082 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583084 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583087 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.591500 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583090 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583093 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583097 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583099 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583102 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583105 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583107 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583110 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583113 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583116 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583118 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583121 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583124 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583126 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583129 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583131 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583134 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583137 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583139 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583142 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.591979 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583144 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583147 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583149 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583152 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583155 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583158 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583162 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583164 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583167 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583169 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583172 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583174 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583177 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583180 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583184 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583188 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583191 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583194 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583198 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.592453 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583201 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583204 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583207 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583210 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583213 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.583215 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.583756 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.590911 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.590927 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.591733 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.591741 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591812 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591818 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591822 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.592940 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591825 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591830 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591834 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591837 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591841 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591844 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591847 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591850 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591853 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591856 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591858 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591861 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591864 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591867 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591870 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591873 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591876 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591879 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591882 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.593293 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591886 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591888 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591892 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591895 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591898 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591901 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591903 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591906 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591909 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591912 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591914 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591917 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591920 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591922 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591925 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591927 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591930 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591932 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591935 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591938 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.593882 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591940 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591943 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591946 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591948 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591951 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591953 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591956 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591958 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591961 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591963 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591965 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591968 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591973 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591976 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591980 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591989 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591992 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591994 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.591997 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592000 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.594391 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592003 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592006 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592009 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592011 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592014 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592016 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592019 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592021 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592024 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592027 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592030 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592032 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592035 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592037 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592040 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592043 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592045 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592048 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592050 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592053 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.594952 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592055 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592058 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592061 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592064 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592066 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592069 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592071 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592074 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592077 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592079 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592082 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592085 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592094 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592096 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592101 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592104 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.595491 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592108 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.592113 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592229 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592234 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592237 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592240 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592243 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592247 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592251 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592255 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592258 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592261 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592264 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592267 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592270 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 20:57:09.595900 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592273 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592276 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592278 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592281 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592284 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592286 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592289 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592291 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592294 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592297 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592300 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592302 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592305 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592308 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592311 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592314 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592316 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592319 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592322 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592324 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 20:57:09.596349 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592327 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592329 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592332 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592334 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592337 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592339 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592342 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592344 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592347 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592350 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592352 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592355 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592357 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592360 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592362 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592365 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592368 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592371 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592373 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592376 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 20:57:09.596917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592379 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592381 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592383 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592387 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592391 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592394 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592397 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592400 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592403 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592406 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592409 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592412 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592414 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592417 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592420 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592423 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592427 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592430 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592433 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 20:57:09.597703 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592436 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592438 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592441 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592443 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592446 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592448 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592451 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592453 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592456 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592459 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592462 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592464 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592467 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592469 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592472 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592475 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592477 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592480 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592483 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 20:57:09.598427 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592485 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592488 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592491 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592493 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592496 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592498 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592501 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:09.592503 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.592509 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.592634 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.594553 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.596093 2567 server.go:1019] "Starting client certificate rotation" Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.596210 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 20:57:09.598909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.596272 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 20:57:09.599436 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.598434 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 20:57:09.601634 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.601612 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 20:57:09.612084 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.612066 2567 log.go:25] "Validated CRI v1 runtime API" Jul 30 20:57:09.614021 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.614007 2567 log.go:25] "Validated CRI v1 image API" Jul 30 20:57:09.616185 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.616170 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 20:57:09.618458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.618436 2567 fs.go:135] Filesystem UUIDs: map[20e0faeb-f71f-446b-804d-91b750701976:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ff3fff99-72ae-44aa-8fc6-0074793b0de9:/dev/nvme0n1p4] Jul 30 20:57:09.618530 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.618457 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 20:57:09.621712 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.621689 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 20:57:09.624330 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.624202 2567 manager.go:216] Machine: {Timestamp:2026-07-30 20:57:09.622236907 +0000 UTC m=+0.321372788 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100306 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a336505c316bd66b46426455be779 SystemUUID:ec2a3365-05c3-16bd-66b4-6426455be779 BootID:64d2f4ef-4cc7-474c-8201-6f671454567a Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:55:2f:82:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:55:2f:82:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:cf:5e:0b:fc:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 20:57:09.624330 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.624324 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 20:57:09.624491 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.624444 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 20:57:09.627096 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.627057 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 20:57:09.627242 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.627098 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-162.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 20:57:09.627734 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.627724 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 20:57:09.627767 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.627735 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 20:57:09.627767 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.627743 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 20:57:09.628404 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.628395 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 20:57:09.629087 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.629077 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 20:57:09.632065 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632053 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 30 20:57:09.632191 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632181 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 20:57:09.632229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632214 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 30 20:57:09.632229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632227 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 20:57:09.632287 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632250 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 20:57:09.632287 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632263 2567 kubelet.go:398] "Adding apiserver pod source" Jul 30 20:57:09.632287 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632274 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 20:57:09.632727 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632716 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 20:57:09.632769 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.632735 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 20:57:09.634955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.634938 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 20:57:09.636399 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.636385 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 20:57:09.636469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.636415 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 20:57:09.637370 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637360 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637375 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637382 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637388 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637398 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637405 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637412 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 20:57:09.637416 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637418 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 20:57:09.637623 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637425 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 20:57:09.637623 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637436 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 20:57:09.637623 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637452 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 20:57:09.637623 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.637457 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 20:57:09.639209 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.639182 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 20:57:09.639209 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.639201 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 20:57:09.641390 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.641368 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-162.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 20:57:09.641390 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.641369 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 20:57:09.641507 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.641405 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 20:57:09.642450 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.642438 2567 server.go:1268] "Started kubelet" Jul 30 20:57:09.642558 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.642510 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 20:57:09.642633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.642547 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 20:57:09.642633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.642603 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 20:57:09.646551 ip-10-0-129-162 systemd[1]: Started Kubernetes Kubelet. Jul 30 20:57:09.646786 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.646717 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 20:57:09.648121 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.647991 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 30 20:57:09.648338 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.648314 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7nckl" Jul 30 20:57:09.649771 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.649750 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 20:57:09.652567 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.652551 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 20:57:09.652567 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.652562 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 20:57:09.652733 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.652717 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:09.652926 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.651455 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-162.ec2.internal.18c72ccb75d29f40 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-162.ec2.internal,UID:ip-10-0-129-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-162.ec2.internal,},FirstTimestamp:2026-07-30 20:57:09.64241184 +0000 UTC m=+0.341547722,LastTimestamp:2026-07-30 20:57:09.64241184 +0000 UTC m=+0.341547722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-162.ec2.internal,}" Jul 30 20:57:09.653078 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653060 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 20:57:09.653193 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653160 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 20:57:09.653193 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653184 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 20:57:09.653342 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653252 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 30 20:57:09.653342 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653262 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 30 20:57:09.653610 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653591 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 20:57:09.653610 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653609 2567 factory.go:55] Registering systemd factory Jul 30 20:57:09.653749 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653620 2567 factory.go:223] Registration of the systemd container factory successfully Jul 30 20:57:09.653844 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653828 2567 factory.go:153] Registering CRI-O factory Jul 30 20:57:09.653844 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653843 2567 factory.go:223] Registration of the crio container factory successfully Jul 30 20:57:09.653969 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653864 2567 factory.go:103] Registering Raw factory Jul 30 20:57:09.653969 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.653877 2567 manager.go:1196] Started watching for new ooms in manager Jul 30 20:57:09.654453 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.654440 2567 manager.go:349] Starting recovery of all containers Jul 30 20:57:09.658132 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.658107 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7nckl" Jul 30 20:57:09.658522 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.658495 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 20:57:09.658626 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.658509 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 20:57:09.664387 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.664230 2567 manager.go:354] Recovery completed Jul 30 20:57:09.665875 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.665792 2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 30 20:57:09.669176 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.669158 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.671005 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.670989 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.671079 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671020 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.671079 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671030 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.671546 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671533 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 20:57:09.671546 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671545 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 20:57:09.671673 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671563 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 30 20:57:09.671673 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.671610 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.672213 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672198 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.672264 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672228 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.672264 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672244 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.672678 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672666 2567 policy_none.go:49] "None policy: Start" Jul 30 20:57:09.672741 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672684 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 20:57:09.672741 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.672713 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 20:57:09.672857 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.672785 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-162.ec2.internal.18c72ccb7786ef66 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-162.ec2.internal,UID:ip-10-0-129-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-162.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-162.ec2.internal,},FirstTimestamp:2026-07-30 20:57:09.671006054 +0000 UTC m=+0.370141935,LastTimestamp:2026-07-30 20:57:09.671006054 +0000 UTC m=+0.370141935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-162.ec2.internal,}" Jul 30 20:57:09.673710 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.673697 2567 policy_none.go:47] "Start" Jul 30 20:57:09.712192 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712173 2567 manager.go:337] "Starting Device Plugin manager" Jul 30 20:57:09.712311 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.712209 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 20:57:09.712311 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712219 2567 server.go:87] "Starting device plugin registration server" Jul 30 20:57:09.712507 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712495 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 20:57:09.712557 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712511 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 20:57:09.712727 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712627 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 20:57:09.712727 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712713 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 20:57:09.712727 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.712722 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 20:57:09.713330 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.713314 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 20:57:09.713415 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.713355 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:09.750268 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.750235 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 20:57:09.751442 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.751425 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 20:57:09.751532 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.751460 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 20:57:09.751532 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.751478 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 20:57:09.751532 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.751508 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 20:57:09.754017 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.754000 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 20:57:09.813014 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.812990 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.813847 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.813832 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.813934 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.813862 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.813934 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.813872 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.813934 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.813895 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.820128 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.820113 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.820196 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.820133 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-162.ec2.internal\": node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:09.834621 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.834601 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:09.852015 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.851992 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal"] Jul 30 20:57:09.852076 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.852053 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.852849 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.852835 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.852930 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.852860 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.852930 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.852870 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.854153 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854138 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.854278 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854262 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.854337 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854301 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.854845 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854832 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.854924 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854847 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.854924 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854855 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.854924 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854865 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.854924 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854871 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.854924 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.854886 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.856041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.856024 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.856102 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.856063 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 20:57:09.856681 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.856665 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientMemory" Jul 30 20:57:09.856755 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.856699 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 20:57:09.856755 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.856715 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeHasSufficientPID" Jul 30 20:57:09.884289 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.884271 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-162.ec2.internal\" not found" node="ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.889030 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.889014 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-162.ec2.internal\" not found" node="ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.934975 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:09.934946 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:09.954740 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.954721 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.954821 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.954744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3cca40a41c5baf92766bc73cc200c75-config\") pod \"kube-apiserver-proxy-ip-10-0-129-162.ec2.internal\" (UID: \"c3cca40a41c5baf92766bc73cc200c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:09.954821 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:09.954762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.035319 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.035288 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.055031 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.054995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.055134 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.055043 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.055134 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.055069 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3cca40a41c5baf92766bc73cc200c75-config\") pod \"kube-apiserver-proxy-ip-10-0-129-162.ec2.internal\" (UID: \"c3cca40a41c5baf92766bc73cc200c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.055134 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.055100 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.055134 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.055108 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615f30ec0b4ece024580b759a8bb64a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal\" (UID: \"615f30ec0b4ece024580b759a8bb64a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.055134 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.055115 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3cca40a41c5baf92766bc73cc200c75-config\") pod \"kube-apiserver-proxy-ip-10-0-129-162.ec2.internal\" (UID: \"c3cca40a41c5baf92766bc73cc200c75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.136538 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.136477 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.188495 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.188441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.191476 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.191461 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:10.237156 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.237125 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.343343 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.343311 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.443689 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.443627 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.544761 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.544723 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.595492 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.595458 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 20:57:10.596051 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.595632 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 20:57:10.645048 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.645023 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.661957 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.661922 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 20:52:10 +0000 UTC" deadline="2028-02-18 16:17:59.934578504 +0000 UTC" Jul 30 20:57:10.661957 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.661953 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13627h20m49.272628802s" Jul 30 20:57:10.723816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.723738 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 20:57:10.745608 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.745568 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.772745 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:10.772711 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615f30ec0b4ece024580b759a8bb64a4.slice/crio-4b7e0b893e2f62897334469a49aaa8a34f6031a3f3f9ac39668c6cb0b9f17019 WatchSource:0}: Error finding container 4b7e0b893e2f62897334469a49aaa8a34f6031a3f3f9ac39668c6cb0b9f17019: Status 404 returned error can't find the container with id 4b7e0b893e2f62897334469a49aaa8a34f6031a3f3f9ac39668c6cb0b9f17019 Jul 30 20:57:10.773166 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:10.773139 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3cca40a41c5baf92766bc73cc200c75.slice/crio-ed8d86f9621447475a1d4316fc10845387e382a62d43cc021841acee7b720db0 WatchSource:0}: Error finding container ed8d86f9621447475a1d4316fc10845387e382a62d43cc021841acee7b720db0: Status 404 returned error can't find the container with id ed8d86f9621447475a1d4316fc10845387e382a62d43cc021841acee7b720db0 Jul 30 20:57:10.777799 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.777783 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 20:57:10.848710 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.848681 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:10.910055 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:10.910034 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 20:57:10.951456 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:10.951433 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:11.056753 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.056674 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-162.ec2.internal\" not found" Jul 30 20:57:11.069461 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.069440 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 20:57:11.154383 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.154356 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" Jul 30 20:57:11.166346 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.166326 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 20:57:11.167252 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.167239 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" Jul 30 20:57:11.175809 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.175788 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 20:57:11.635572 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.635536 2567 apiserver.go:52] "Watching apiserver" Jul 30 20:57:11.641378 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.641348 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 20:57:11.643483 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.643458 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vg98k","kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal","openshift-dns/node-resolver-45ms9","openshift-multus/multus-additional-cni-plugins-zjfqt","openshift-multus/network-metrics-daemon-qct22","openshift-network-diagnostics/network-check-target-w5rlj","openshift-network-operator/iptables-alerter-6dtk9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb","openshift-cluster-node-tuning-operator/tuned-t29wp","openshift-image-registry/node-ca-nk9g2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal","openshift-multus/multus-84p8s","openshift-ovn-kubernetes/ovnkube-node-sj7bz","kube-system/global-pull-secret-syncer-ptckn"] Jul 30 20:57:11.645230 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.645149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.646771 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.646753 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.647980 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.647948 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 20:57:11.648230 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.648212 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.648406 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.648389 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.648599 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.648583 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nn8jd\"" Jul 30 20:57:11.650061 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.648800 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.650061 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.649398 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 20:57:11.650061 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.649833 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 20:57:11.651762 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.650047 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 20:57:11.651762 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.651313 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qn2dq\"" Jul 30 20:57:11.651762 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.651535 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652069 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652094 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652138 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652288 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxxj6\"" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.652321 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652564 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.652617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652582 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:11.653094 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.652640 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:11.653094 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.652720 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.653379 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.653354 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 20:57:11.654145 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.653917 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 20:57:11.654721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654379 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.654721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 20:57:11.654721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654510 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-br6zp\"" Jul 30 20:57:11.654721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654593 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.654721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654670 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 20:57:11.655033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.654920 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.655033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.655008 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.656779 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.656758 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.657648 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.657628 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 20:57:11.657861 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.657839 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.658053 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.658040 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 20:57:11.658278 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.658226 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4gggh\"" Jul 30 20:57:11.658486 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.658452 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7qb6j\"" Jul 30 20:57:11.658762 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.658723 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.659079 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.658782 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:11.659990 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.659870 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.660621 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.660600 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.661981 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.661963 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.662367 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.662348 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8kpvk\"" Jul 30 20:57:11.662447 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.662438 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 20:57:11.662780 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.662761 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 20:57:11.663032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.662998 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h7524\"" Jul 30 20:57:11.663100 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.663044 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.663257 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.663239 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 20:57:11.663349 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.663329 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 20:57:11.665446 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.665422 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.666817 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-systemd\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.666939 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666835 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-tmp\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.666939 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666871 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqbg\" (UniqueName: \"kubernetes.io/projected/f8ad8626-c218-4c3f-918d-4295881c08e3-kube-api-access-cxqbg\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.666939 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666897 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-log-socket\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.666939 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666925 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667153 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666953 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-hosts-file\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.667153 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.666985 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-lib-modules\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.667258 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667175 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bjk\" (UniqueName: \"kubernetes.io/projected/70417306-a8ec-4183-830b-4de0b493432e-kube-api-access-l4bjk\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.667320 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667249 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.667320 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667287 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-systemd-units\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667430 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667321 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-ovn\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667430 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-config\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667430 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667408 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-env-overrides\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667595 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysconfig\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.667595 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667480 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-etc-tuned\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.667595 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cnibin\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.667595 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667572 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-systemd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667693 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-node-log\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667724 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wdxf\" (UniqueName: \"kubernetes.io/projected/a3f68ca9-ee11-4230-a031-dec66f3424e6-kube-api-access-5wdxf\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-tmp-dir\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667801 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-var-lib-kubelet\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.667846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1232d4ba-374c-48ee-aa85-647959e07556-agent-certs\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.668143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667928 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.668143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.667958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwpdq\" (UniqueName: \"kubernetes.io/projected/2a37f37e-1a22-459b-ad85-aa6869c43d5c-kube-api-access-hwpdq\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.668143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-kubelet\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.668143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-etc-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.668143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovn-node-metrics-cert\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.668810 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-system-cni-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.668922 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668876 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-modprobe-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.668922 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.669033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.669033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-bin\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.668990 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-host-slash\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.669033 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669014 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-netns\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669235 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669125 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669235 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-sys\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.669469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669401 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1232d4ba-374c-48ee-aa85-647959e07556-konnectivity-ca\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.669469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669458 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-os-release\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.669607 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669488 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.669607 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669564 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669715 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-netd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669715 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669659 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-script-lib\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.669715 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcqfq\" (UniqueName: \"kubernetes.io/projected/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-kube-api-access-gcqfq\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.669865 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669735 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-kubernetes\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.669865 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669790 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.669865 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-run\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.670011 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669895 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-host\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.670011 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669919 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-iptables-alerter-script\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.670011 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.669979 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-slash\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.670011 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670004 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-var-lib-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.670219 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670060 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrvx\" (UniqueName: \"kubernetes.io/projected/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-kube-api-access-8jrvx\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.670219 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670088 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-conf\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.671133 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670922 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 20:57:11.671133 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670924 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 20:57:11.671133 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.670980 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k7l4d\"" Jul 30 20:57:11.685279 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.685258 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gj9m8" Jul 30 20:57:11.691906 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.691888 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gj9m8" Jul 30 20:57:11.754233 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.754211 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 20:57:11.759169 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.758859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" event={"ID":"615f30ec0b4ece024580b759a8bb64a4","Type":"ContainerStarted","Data":"4b7e0b893e2f62897334469a49aaa8a34f6031a3f3f9ac39668c6cb0b9f17019"} Jul 30 20:57:11.761788 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.761129 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" event={"ID":"c3cca40a41c5baf92766bc73cc200c75","Type":"ContainerStarted","Data":"ed8d86f9621447475a1d4316fc10845387e382a62d43cc021841acee7b720db0"} Jul 30 20:57:11.770302 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-modprobe-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770391 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770317 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770391 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-bin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770391 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770369 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-conf-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770396 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770421 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-dbus\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770445 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-device-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-sys\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1232d4ba-374c-48ee-aa85-647959e07556-konnectivity-ca\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770549 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770577 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-registration-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.770616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770603 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-hostroot\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-multus-certs\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770681 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcqfq\" (UniqueName: \"kubernetes.io/projected/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-kube-api-access-gcqfq\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770706 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-run\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-iptables-alerter-script\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770779 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-slash\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-log-socket\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-etc-selinux\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770854 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-system-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770880 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-config\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-script-lib\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770919 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-conf\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770934 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-tmp\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770949 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqbg\" (UniqueName: \"kubernetes.io/projected/f8ad8626-c218-4c3f-918d-4295881c08e3-kube-api-access-cxqbg\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770977 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghj4\" (UniqueName: \"kubernetes.io/projected/d025f885-6bc9-4f24-b49d-9604246fea7f-kube-api-access-9ghj4\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.770992 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.770994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-lib-modules\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771026 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-systemd-units\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771089 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-ovn\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysconfig\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-etc-tuned\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771158 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/597f0712-04ff-4ba3-b59d-5df371cd7a8f-serviceca\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771157 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-modprobe-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771215 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-tmp-dir\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771383 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771411 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-kubelet\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-system-cni-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771502 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.771785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-bin\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-kubelet-config\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771585 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-socket-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771602 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-multus\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771641 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-host-slash\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771642 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysconfig\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-netns\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771705 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsdnq\" (UniqueName: \"kubernetes.io/projected/597f0712-04ff-4ba3-b59d-5df371cd7a8f-kube-api-access-bsdnq\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771731 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-cnibin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771738 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-d\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771758 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-os-release\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771787 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771790 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-sys\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-netd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-kubelet\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771868 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-kubernetes\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-host\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.772594 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.771967 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-lib-modules\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-var-lib-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772754 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-etc-kubernetes\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772785 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcf5f\" (UniqueName: \"kubernetes.io/projected/3db6b84c-1324-41f9-a09a-a73131477031-kube-api-access-xcf5f\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772815 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-env-overrides\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772843 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrvx\" (UniqueName: \"kubernetes.io/projected/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-kube-api-access-8jrvx\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772876 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-tmp-dir\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-systemd\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772948 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-kubernetes\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772965 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-systemd\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.772994 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/597f0712-04ff-4ba3-b59d-5df371cd7a8f-host\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773010 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-host\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-hosts-file\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773041 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-hosts-file\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bjk\" (UniqueName: \"kubernetes.io/projected/70417306-a8ec-4183-830b-4de0b493432e-kube-api-access-l4bjk\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773067 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.773420 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773074 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773080 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773095 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-slash\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-sys-fs\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773120 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-log-socket\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-os-release\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773165 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-socket-dir-parent\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-k8s-cni-cncf-io\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773223 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-netns\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cnibin\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773300 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-systemd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773329 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-node-log\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773490 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-config\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773522 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cnibin\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773563 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-netd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773568 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-systemd\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.774272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773598 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-node-log\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773613 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773635 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-os-release\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773647 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773677 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-cni-bin\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773716 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-var-lib-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773736 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a37f37e-1a22-459b-ad85-aa6869c43d5c-system-cni-dir\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773752 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-host-slash\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wdxf\" (UniqueName: \"kubernetes.io/projected/a3f68ca9-ee11-4230-a031-dec66f3424e6-kube-api-access-5wdxf\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773792 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-run-netns\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-multus-daemon-config\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773828 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-var-lib-kubelet\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1232d4ba-374c-48ee-aa85-647959e07556-agent-certs\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwpdq\" (UniqueName: \"kubernetes.io/projected/2a37f37e-1a22-459b-ad85-aa6869c43d5c-kube-api-access-hwpdq\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-etc-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovn-node-metrics-cert\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.773937 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-cni-binary-copy\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.775115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774030 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-var-lib-kubelet\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774027 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-env-overrides\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774177 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-run\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.774210 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774216 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-systemd-units\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774249 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-etc-openvswitch\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.774293 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:12.274254012 +0000 UTC m=+2.973389897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774786 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1232d4ba-374c-48ee-aa85-647959e07556-konnectivity-ca\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovnkube-script-lib\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.774998 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-host-kubelet\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.775036 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3f68ca9-ee11-4230-a031-dec66f3424e6-run-ovn\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.775311 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a37f37e-1a22-459b-ad85-aa6869c43d5c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.775355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-iptables-alerter-script\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.777041 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.775428 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70417306-a8ec-4183-830b-4de0b493432e-etc-sysctl-conf\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.778087 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.777857 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-tmp\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.778087 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.777949 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3f68ca9-ee11-4230-a031-dec66f3424e6-ovn-node-metrics-cert\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.778087 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.778013 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70417306-a8ec-4183-830b-4de0b493432e-etc-tuned\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.780193 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.780112 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1232d4ba-374c-48ee-aa85-647959e07556-agent-certs\") pod \"konnectivity-agent-vg98k\" (UID: \"1232d4ba-374c-48ee-aa85-647959e07556\") " pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:11.782389 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.781240 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqbg\" (UniqueName: \"kubernetes.io/projected/f8ad8626-c218-4c3f-918d-4295881c08e3-kube-api-access-cxqbg\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:11.787324 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.785332 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcqfq\" (UniqueName: \"kubernetes.io/projected/8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4-kube-api-access-gcqfq\") pod \"node-resolver-45ms9\" (UID: \"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4\") " pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.787795 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.787550 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrvx\" (UniqueName: \"kubernetes.io/projected/5c1b7d55-864c-4bf3-ad68-3c7652fb59b1-kube-api-access-8jrvx\") pod \"iptables-alerter-6dtk9\" (UID: \"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1\") " pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.788383 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.787975 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bjk\" (UniqueName: \"kubernetes.io/projected/70417306-a8ec-4183-830b-4de0b493432e-kube-api-access-l4bjk\") pod \"tuned-t29wp\" (UID: \"70417306-a8ec-4183-830b-4de0b493432e\") " pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:11.788383 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.788195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwpdq\" (UniqueName: \"kubernetes.io/projected/2a37f37e-1a22-459b-ad85-aa6869c43d5c-kube-api-access-hwpdq\") pod \"multus-additional-cni-plugins-zjfqt\" (UID: \"2a37f37e-1a22-459b-ad85-aa6869c43d5c\") " pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:11.788383 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.788322 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:11.788383 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.788369 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:11.788383 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.788385 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:11.788751 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.788441 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:12.288424504 +0000 UTC m=+2.987560396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:11.792721 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.792685 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wdxf\" (UniqueName: \"kubernetes.io/projected/a3f68ca9-ee11-4230-a031-dec66f3424e6-kube-api-access-5wdxf\") pod \"ovnkube-node-sj7bz\" (UID: \"a3f68ca9-ee11-4230-a031-dec66f3424e6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.875239 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-etc-kubernetes\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875245 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcf5f\" (UniqueName: \"kubernetes.io/projected/3db6b84c-1324-41f9-a09a-a73131477031-kube-api-access-xcf5f\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875274 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/597f0712-04ff-4ba3-b59d-5df371cd7a8f-host\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875298 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875323 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-etc-kubernetes\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-sys-fs\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875388 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-sys-fs\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.875417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875391 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-os-release\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875420 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-socket-dir-parent\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875443 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-k8s-cni-cncf-io\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875478 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-netns\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-k8s-cni-cncf-io\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875483 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/597f0712-04ff-4ba3-b59d-5df371cd7a8f-host\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875527 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-multus-daemon-config\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-cni-binary-copy\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875560 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-os-release\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875561 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-netns\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-socket-dir-parent\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.875806 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875796 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-bin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-conf-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875885 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-multus-conf-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875894 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-bin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875912 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-dbus\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.875955 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-device-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876000 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-registration-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876025 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-dbus\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876023 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-device-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876027 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-hostroot\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-multus-certs\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876081 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-registration-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876095 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-etc-selinux\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-system-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876127 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-hostroot\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghj4\" (UniqueName: \"kubernetes.io/projected/d025f885-6bc9-4f24-b49d-9604246fea7f-kube-api-access-9ghj4\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-multus-daemon-config\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.876458 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876170 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-run-multus-certs\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/597f0712-04ff-4ba3-b59d-5df371cd7a8f-serviceca\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876220 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-system-cni-dir\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876255 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-etc-selinux\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.876339 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-kubelet-config\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:11.876409 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:12.376392807 +0000 UTC m=+3.075528681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-socket-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876448 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/53886e58-f91a-4e1f-a5a2-331f73bc8668-kubelet-config\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876465 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-multus\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsdnq\" (UniqueName: \"kubernetes.io/projected/597f0712-04ff-4ba3-b59d-5df371cd7a8f-kube-api-access-bsdnq\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876518 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-cnibin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-kubelet\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876577 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d025f885-6bc9-4f24-b49d-9604246fea7f-socket-dir\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876613 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-kubelet\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876673 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-host-var-lib-cni-multus\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.877362 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3db6b84c-1324-41f9-a09a-a73131477031-cnibin\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.878220 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.876724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/597f0712-04ff-4ba3-b59d-5df371cd7a8f-serviceca\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.878220 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.877139 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3db6b84c-1324-41f9-a09a-a73131477031-cni-binary-copy\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.885312 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.885287 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghj4\" (UniqueName: \"kubernetes.io/projected/d025f885-6bc9-4f24-b49d-9604246fea7f-kube-api-access-9ghj4\") pod \"aws-ebs-csi-driver-node-cg5nb\" (UID: \"d025f885-6bc9-4f24-b49d-9604246fea7f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:11.886036 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.885958 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsdnq\" (UniqueName: \"kubernetes.io/projected/597f0712-04ff-4ba3-b59d-5df371cd7a8f-kube-api-access-bsdnq\") pod \"node-ca-nk9g2\" (UID: \"597f0712-04ff-4ba3-b59d-5df371cd7a8f\") " pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:11.886519 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.886492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcf5f\" (UniqueName: \"kubernetes.io/projected/3db6b84c-1324-41f9-a09a-a73131477031-kube-api-access-xcf5f\") pod \"multus-84p8s\" (UID: \"3db6b84c-1324-41f9-a09a-a73131477031\") " pod="openshift-multus/multus-84p8s" Jul 30 20:57:11.974692 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.974654 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6dtk9" Jul 30 20:57:11.983291 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.983240 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:11.994082 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.994055 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-45ms9" Jul 30 20:57:11.998846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:11.998807 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" Jul 30 20:57:12.008573 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.008554 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t29wp" Jul 30 20:57:12.017204 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.017185 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:12.026831 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.026811 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" Jul 30 20:57:12.035611 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.035592 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nk9g2" Jul 30 20:57:12.042404 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.042385 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-84p8s" Jul 30 20:57:12.167338 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.167260 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 20:57:12.278844 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.278811 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:12.279010 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.278944 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:12.279010 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.279004 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:13.278988659 +0000 UTC m=+3.978124529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:12.379260 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.379221 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:12.379437 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.379268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:12.379437 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379393 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:12.379556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379450 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:12.379556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379467 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:13.379446263 +0000 UTC m=+4.078582154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:12.379556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379477 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:12.379556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379491 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:12.379556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:12.379556 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:13.379539255 +0000 UTC m=+4.078675144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:12.694772 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.694728 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 20:52:12 +0000 UTC" deadline="2028-03-29 21:15:42.352288849 +0000 UTC" Jul 30 20:57:12.694772 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:12.694770 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14592h18m29.657524081s" Jul 30 20:57:12.972420 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:12.972393 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db6b84c_1324_41f9_a09a_a73131477031.slice/crio-fb2b867a6beb233a10f468bd3de3ff35f2050913a6bcfc6eeca6b84042141dac WatchSource:0}: Error finding container fb2b867a6beb233a10f468bd3de3ff35f2050913a6bcfc6eeca6b84042141dac: Status 404 returned error can't find the container with id fb2b867a6beb233a10f468bd3de3ff35f2050913a6bcfc6eeca6b84042141dac Jul 30 20:57:12.974749 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:12.974725 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f68ca9_ee11_4230_a031_dec66f3424e6.slice/crio-e2e63285a40301b7b193980de9bb02639923f42d291f6d5e5ae6c077e823994d WatchSource:0}: Error finding container e2e63285a40301b7b193980de9bb02639923f42d291f6d5e5ae6c077e823994d: Status 404 returned error can't find the container with id e2e63285a40301b7b193980de9bb02639923f42d291f6d5e5ae6c077e823994d Jul 30 20:57:13.285730 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.285694 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:13.285869 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.285851 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:13.285924 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.285914 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:15.285899053 +0000 UTC m=+5.985034935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.386868 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.386941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387077 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387144 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:15.387125408 +0000 UTC m=+6.086261280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387411 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387434 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387448 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:13.387516 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.387503 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:15.387486089 +0000 UTC m=+6.086621968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:13.695889 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.695822 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 20:52:12 +0000 UTC" deadline="2028-01-03 12:09:44.259063486 +0000 UTC" Jul 30 20:57:13.695889 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.695886 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12519h12m30.563182176s" Jul 30 20:57:13.754560 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.754493 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:13.754742 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.754637 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:13.754811 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.754775 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:13.755431 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.755204 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:13.755431 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.755311 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:13.755833 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:13.755793 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:13.788734 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.788680 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" event={"ID":"d025f885-6bc9-4f24-b49d-9604246fea7f","Type":"ContainerStarted","Data":"55b0df6ade1b8a7bb73b49dd41ef8e557d8e25f2d344480a36719bb3a3acd6e6"} Jul 30 20:57:13.808759 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.808673 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t29wp" event={"ID":"70417306-a8ec-4183-830b-4de0b493432e","Type":"ContainerStarted","Data":"c63c3f46d0dfd8966cbdb788ff1519199d570c88c4d1cde4705b7b2d191a9645"} Jul 30 20:57:13.814519 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.814488 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerStarted","Data":"c9aa8dd6bedf2aee1e7545536ac0282404f846675dcb046e634d371774766a08"} Jul 30 20:57:13.829491 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.829460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-84p8s" event={"ID":"3db6b84c-1324-41f9-a09a-a73131477031","Type":"ContainerStarted","Data":"fb2b867a6beb233a10f468bd3de3ff35f2050913a6bcfc6eeca6b84042141dac"} Jul 30 20:57:13.846718 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.846676 2567 generic.go:356] "Generic (PLEG): container finished" podID="615f30ec0b4ece024580b759a8bb64a4" containerID="507d28c22b4239bfbf4b616dcd22e12cccb1a180086306033f67bc36372a8355" exitCode=0 Jul 30 20:57:13.846848 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.846764 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" event={"ID":"615f30ec0b4ece024580b759a8bb64a4","Type":"ContainerDied","Data":"507d28c22b4239bfbf4b616dcd22e12cccb1a180086306033f67bc36372a8355"} Jul 30 20:57:13.856404 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.856374 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vg98k" event={"ID":"1232d4ba-374c-48ee-aa85-647959e07556","Type":"ContainerStarted","Data":"456e5a90aec2d0a53045542e45975260957e40744fe16118d8350e1131922763"} Jul 30 20:57:13.872939 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.872905 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-45ms9" event={"ID":"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4","Type":"ContainerStarted","Data":"ff47509cf52fd946e7d48ec0f70be9e39620220e4c3174a0400f3202a5feb778"} Jul 30 20:57:13.903501 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.903468 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6dtk9" event={"ID":"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1","Type":"ContainerStarted","Data":"5b0a5999176b1c3335ea7823a882f5a658c6d5b1cb897ccb5ad60df7d6c8ff29"} Jul 30 20:57:13.927503 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.927473 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nk9g2" event={"ID":"597f0712-04ff-4ba3-b59d-5df371cd7a8f","Type":"ContainerStarted","Data":"cdda337f0323fbcf12979571facb0be072db8ca66a09d01032bbd16779431e08"} Jul 30 20:57:13.931699 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.931510 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"e2e63285a40301b7b193980de9bb02639923f42d291f6d5e5ae6c077e823994d"} Jul 30 20:57:13.945192 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:13.945165 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" event={"ID":"c3cca40a41c5baf92766bc73cc200c75","Type":"ContainerStarted","Data":"ac73bffb76bd7c8f7ba2486245bcc65915c93f0460ce2360957d4d23ad647d01"} Jul 30 20:57:14.965973 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:14.965738 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" event={"ID":"615f30ec0b4ece024580b759a8bb64a4","Type":"ContainerStarted","Data":"effecbf5db5917b342dac5ceb9ed210df729dfd46aa5b59ddf7d8c82661fba99"} Jul 30 20:57:14.982428 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:14.982364 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-162.ec2.internal" podStartSLOduration=2.9823431019999997 podStartE2EDuration="2.982343102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:14.98157453 +0000 UTC m=+5.680710424" watchObservedRunningTime="2026-07-30 20:57:14.982343102 +0000 UTC m=+5.681478997" Jul 30 20:57:14.982935 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:14.982897 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-162.ec2.internal" podStartSLOduration=2.982880106 podStartE2EDuration="2.982880106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:13.959935329 +0000 UTC m=+4.659071223" watchObservedRunningTime="2026-07-30 20:57:14.982880106 +0000 UTC m=+5.682016000" Jul 30 20:57:15.307422 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.303671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:15.307422 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.303870 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:15.307422 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.303938 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:19.303915431 +0000 UTC m=+10.003051316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:15.404648 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.404608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:15.404648 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.404664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:15.404899 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.404796 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:15.404899 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.404859 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:19.40484029 +0000 UTC m=+10.103976175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:15.405305 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.405283 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:15.405404 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.405311 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:15.405404 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.405326 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:15.405404 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.405375 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:19.405357506 +0000 UTC m=+10.104493383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:15.754467 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.754390 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:15.754628 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.754544 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:15.754955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.754936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:15.755082 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.755061 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:15.755147 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:15.755114 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:15.755203 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:15.755187 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:17.754501 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:17.753784 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:17.754501 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:17.753942 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:17.754501 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:17.754350 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:17.754501 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:17.754449 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:17.755251 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:17.755221 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:17.755383 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:17.755323 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:19.338993 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.338950 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:19.339448 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.339119 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:19.339448 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.339197 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:27.339173806 +0000 UTC m=+18.038309680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.439442 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.439493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.439634 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.439702 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:27.439682822 +0000 UTC m=+18.138818711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.440130 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.440149 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.440163 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:19.440270 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.440211 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:27.440194304 +0000 UTC m=+18.139330177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:19.754601 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.752745 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:19.754601 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.752891 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:19.754601 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.753266 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:19.754601 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.753355 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:19.755138 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:19.754994 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:19.755138 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:19.755100 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:21.752488 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:21.752441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:21.752488 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:21.752490 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:21.753085 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:21.752441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:21.753085 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:21.752584 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:21.753085 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:21.752753 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:21.753085 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:21.752840 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:23.751933 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:23.751903 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:23.752389 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:23.751903 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:23.752389 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:23.752031 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:23.752389 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:23.752077 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:23.752389 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:23.751901 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:23.752389 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:23.752162 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:25.752345 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:25.752311 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:25.752345 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:25.752337 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:25.752852 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:25.752319 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:25.752852 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:25.752435 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:25.752852 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:25.752532 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:25.752852 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:25.752659 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:27.399457 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.399417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:27.400020 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.399597 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:27.400020 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.399677 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:43.399654684 +0000 UTC m=+34.098790553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:27.500094 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.500061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:27.500245 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.500105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:27.500245 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500222 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:27.500245 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500229 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:27.500373 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500252 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:27.500373 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500268 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:27.500373 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500284 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:43.500266463 +0000 UTC m=+34.199402333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:27.500373 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.500308 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:57:43.500296403 +0000 UTC m=+34.199432271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:27.752487 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.752452 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:27.752671 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.752452 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:27.752671 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.752592 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:27.752789 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.752677 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:27.752789 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:27.752467 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:27.752789 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:27.752766 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:29.753306 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:29.753271 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:29.753703 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:29.753359 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:29.753703 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:29.753375 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:29.753703 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:29.753407 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:29.753703 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:29.753437 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:29.753703 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:29.753500 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:31.752165 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:31.752132 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:31.752633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:31.752132 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:31.752633 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:31.752262 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:31.752633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:31.752149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:31.752633 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:31.752321 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:31.752633 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:31.752382 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:33.004373 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.004008 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-84p8s" event={"ID":"3db6b84c-1324-41f9-a09a-a73131477031","Type":"ContainerStarted","Data":"37c28ceafa6ec038a1d7408c3154616557cb1507129c6c34dd5f561579a92b6e"} Jul 30 20:57:33.005491 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.005472 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vg98k" event={"ID":"1232d4ba-374c-48ee-aa85-647959e07556","Type":"ContainerStarted","Data":"234b4d6050cce33dcf8b9b702d4ca4c55d7539cf2f1738f64a31b1a17352c25b"} Jul 30 20:57:33.009516 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.009488 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-45ms9" event={"ID":"8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4","Type":"ContainerStarted","Data":"cd21bb38e9c4c14d0aad9d987b065faaea4addbc3670c378fe3eea943628cf3d"} Jul 30 20:57:33.010828 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.010807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nk9g2" event={"ID":"597f0712-04ff-4ba3-b59d-5df371cd7a8f","Type":"ContainerStarted","Data":"d81a60a84d5f256972732396c1b49101c5360262ebfd9e020ec1fcaf593fdc21"} Jul 30 20:57:33.013510 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013489 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"5b750a172f5fe742a5af13a31b4e448db49b4d00a0e57e1f353562ec64a0e75e"} Jul 30 20:57:33.013617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013545 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"88a0eb42ec9b528353f9a3d51689aaa4f954fb2272ed8832c898f7c2d550399b"} Jul 30 20:57:33.013617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013573 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"15300c8d53520d4e92fa21e4e7a64d1d7856cd2e043b646b5d99a9d1cf97b45b"} Jul 30 20:57:33.013617 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"0eae5a01c40b0c763403b1cf62fdf98c366716c909e9d8c162d8b3fa60008150"} Jul 30 20:57:33.013769 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013644 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"1759d777257ae4cfd5d9867b3b6463a319e45f666550c294f758c79c1f92e87c"} Jul 30 20:57:33.013769 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.013677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"32c8a97242501d591da3d389bed591edfda142f30170f8adce29db6af4455d93"} Jul 30 20:57:33.014776 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.014748 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" event={"ID":"d025f885-6bc9-4f24-b49d-9604246fea7f","Type":"ContainerStarted","Data":"ac2753ebcfaaee6ba6d6c915cb9d76dac17e0840d9f04565770c1737257b315a"} Jul 30 20:57:33.015943 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.015924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t29wp" event={"ID":"70417306-a8ec-4183-830b-4de0b493432e","Type":"ContainerStarted","Data":"bc9b21fe2f3618a2fd09bda79dc82d2c014ca321aa3fd90360e30a7aaeee9bd0"} Jul 30 20:57:33.017234 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.017213 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="ad92cc53808d2bd3cf0347b6da402de12b532366e880a2377d66de452248a2a1" exitCode=0 Jul 30 20:57:33.017334 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.017241 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"ad92cc53808d2bd3cf0347b6da402de12b532366e880a2377d66de452248a2a1"} Jul 30 20:57:33.020540 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.020496 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-84p8s" podStartSLOduration=3.727856519 podStartE2EDuration="23.020460163s" totalImagesPullingTime="19.292603644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:33.020171617 +0000 UTC m=+23.719307508" watchObservedRunningTime="2026-07-30 20:57:33.020460163 +0000 UTC m=+23.719596055" Jul 30 20:57:33.034967 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.034931 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t29wp" podStartSLOduration=3.828426219 podStartE2EDuration="23.034920411s" totalImagesPullingTime="19.206494192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:33.034698945 +0000 UTC m=+23.733834834" watchObservedRunningTime="2026-07-30 20:57:33.034920411 +0000 UTC m=+23.734056301" Jul 30 20:57:33.048711 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.048679 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vg98k" podStartSLOduration=3.840762821 podStartE2EDuration="23.048667875s" totalImagesPullingTime="19.207905054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:33.048524964 +0000 UTC m=+23.747660854" watchObservedRunningTime="2026-07-30 20:57:33.048667875 +0000 UTC m=+23.747803764" Jul 30 20:57:33.062441 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.062410 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nk9g2" podStartSLOduration=3.845054869 podStartE2EDuration="23.06240002s" totalImagesPullingTime="19.217345151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:33.062136541 +0000 UTC m=+23.761272443" watchObservedRunningTime="2026-07-30 20:57:33.06240002 +0000 UTC m=+23.761535909" Jul 30 20:57:33.076136 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.076101 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-45ms9" podStartSLOduration=3.8585415259999998 podStartE2EDuration="23.07609135s" totalImagesPullingTime="19.217549824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:33.075753007 +0000 UTC m=+23.774888898" watchObservedRunningTime="2026-07-30 20:57:33.07609135 +0000 UTC m=+23.775227240" Jul 30 20:57:33.752254 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.752226 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:33.752451 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.752226 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:33.752451 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:33.752340 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:33.752451 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:33.752337 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:33.752451 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:33.752413 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:33.752682 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:33.752476 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:34.004031 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.004003 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 20:57:34.021732 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.021267 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6dtk9" event={"ID":"5c1b7d55-864c-4bf3-ad68-3c7652fb59b1","Type":"ContainerStarted","Data":"00377791428e020cd8f89a6b315fd5994b9b59e8f00affc35a99372898423ce9"} Jul 30 20:57:34.023264 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.023239 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" event={"ID":"d025f885-6bc9-4f24-b49d-9604246fea7f","Type":"ContainerStarted","Data":"6c4a25d531d5154a847c9c5b1a62a0be524ef12a2b50703d84e0924ce04b64ac"} Jul 30 20:57:34.052923 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.052881 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6dtk9" podStartSLOduration=4.842106115 podStartE2EDuration="24.052864288s" totalImagesPullingTime="19.210758173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:34.052484266 +0000 UTC m=+24.751620156" watchObservedRunningTime="2026-07-30 20:57:34.052864288 +0000 UTC m=+24.752000180" Jul 30 20:57:34.729129 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.729025 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T20:57:34.004027833Z","UUID":"d985ea9a-c219-4ba5-b330-7e87eb36ab75","Handler":null,"Name":"","Endpoint":""} Jul 30 20:57:34.730763 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.730742 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 20:57:34.730889 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:34.730777 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 20:57:35.273818 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:35.273598 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:35.274312 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:35.274279 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:35.755926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:35.755900 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:35.756127 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:35.755944 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:35.756127 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:35.755911 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:35.756127 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:35.756027 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:35.756127 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:35.756119 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:35.756315 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:35.756200 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:36.030814 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:36.030731 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" event={"ID":"d025f885-6bc9-4f24-b49d-9604246fea7f","Type":"ContainerStarted","Data":"b8bd47792f120302c4d75c752ae5de68ea964722b2fdb07188a281e8aa017d54"} Jul 30 20:57:36.034454 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:36.034372 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"6c7e69f76865354fbdee3e782b77dbd9f8e130af04949f0d5a4f03d2798b48cd"} Jul 30 20:57:36.034633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:36.034608 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:36.035068 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:36.035048 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vg98k" Jul 30 20:57:36.048706 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:36.048661 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cg5nb" podStartSLOduration=4.37061048 podStartE2EDuration="26.048644716s" totalImagesPullingTime="21.678034236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:36.048085556 +0000 UTC m=+26.747221446" watchObservedRunningTime="2026-07-30 20:57:36.048644716 +0000 UTC m=+26.747780608" Jul 30 20:57:37.752451 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:37.752275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:37.753148 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:37.752275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:37.753148 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:37.752540 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:37.753148 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:37.752291 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:37.753148 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:37.752602 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:37.753148 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:37.752678 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:38.040224 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.040191 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="700af02e5933f350675b3cafda2c5ddd9559a820149ab3340e79b893c665008c" exitCode=0 Jul 30 20:57:38.040365 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.040277 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"700af02e5933f350675b3cafda2c5ddd9559a820149ab3340e79b893c665008c"} Jul 30 20:57:38.044078 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.044057 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" event={"ID":"a3f68ca9-ee11-4230-a031-dec66f3424e6","Type":"ContainerStarted","Data":"9ef83c750cbbef120c75c54d0c4d941030272fe952fdbc2a0b791e5cdb88aabd"} Jul 30 20:57:38.044367 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.044344 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:38.044491 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.044414 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:38.044491 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.044463 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:38.059526 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.059508 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:38.059662 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.059636 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:57:38.085631 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:38.085575 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" podStartSLOduration=8.800906136 podStartE2EDuration="28.085563773s" totalImagesPullingTime="19.284657637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:38.084557105 +0000 UTC m=+28.783692991" watchObservedRunningTime="2026-07-30 20:57:38.085563773 +0000 UTC m=+28.784699663" Jul 30 20:57:39.659874 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.659481 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qct22"] Jul 30 20:57:39.660385 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.659882 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:39.660385 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:39.660009 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:39.660385 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.660332 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ptckn"] Jul 30 20:57:39.660552 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.660415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:39.660552 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:39.660492 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:39.661925 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.661904 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w5rlj"] Jul 30 20:57:39.662031 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:39.661984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:39.662092 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:39.662073 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:40.050879 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:40.050845 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="4004162c3dc9c371455162b24aac20802f19602d18fe010c5e34a53444efb92e" exitCode=0 Jul 30 20:57:40.051075 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:40.050922 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"4004162c3dc9c371455162b24aac20802f19602d18fe010c5e34a53444efb92e"} Jul 30 20:57:40.752688 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:40.752653 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:40.753228 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:40.752799 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:41.752092 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:41.752059 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:41.752257 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:41.752109 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:41.752257 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:41.752203 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:41.752373 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:41.752305 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:42.058517 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:42.058487 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="f3d358cf27d31335d92e5fae8093296144c39a7579ac5173d1ee80d983676e26" exitCode=0 Jul 30 20:57:42.059195 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:42.058549 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"f3d358cf27d31335d92e5fae8093296144c39a7579ac5173d1ee80d983676e26"} Jul 30 20:57:42.752053 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:42.752015 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:42.752214 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:42.752140 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:43.415920 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:43.415837 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:43.416356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.415936 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:43.416356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.415991 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:15.415975603 +0000 UTC m=+66.115111474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 20:57:43.516843 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:43.516813 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:43.517026 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:43.516851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:43.517026 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.516976 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 20:57:43.517026 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.516987 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:43.517026 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.517000 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 20:57:43.517026 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.517012 2567 projected.go:196] Error preparing data for projected volume kube-api-access-dmqrq for pod openshift-network-diagnostics/network-check-target-w5rlj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:43.517236 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.517043 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret podName:53886e58-f91a-4e1f-a5a2-331f73bc8668 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:15.517025815 +0000 UTC m=+66.216161685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret") pod "global-pull-secret-syncer-ptckn" (UID: "53886e58-f91a-4e1f-a5a2-331f73bc8668") : object "kube-system"/"original-pull-secret" not registered Jul 30 20:57:43.517236 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.517060 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq podName:3b462055-f414-4a97-ac4c-9a8e02a889e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:15.517051976 +0000 UTC m=+66.216187845 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmqrq" (UniqueName: "kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq") pod "network-check-target-w5rlj" (UID: "3b462055-f414-4a97-ac4c-9a8e02a889e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 20:57:43.758541 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:43.758514 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:43.758729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:43.758514 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:43.758729 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.758650 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptckn" podUID="53886e58-f91a-4e1f-a5a2-331f73bc8668" Jul 30 20:57:43.758729 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:43.758692 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5rlj" podUID="3b462055-f414-4a97-ac4c-9a8e02a889e0" Jul 30 20:57:44.751826 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:44.751801 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:44.752194 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:44.751944 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qct22" podUID="f8ad8626-c218-4c3f-918d-4295881c08e3" Jul 30 20:57:45.130483 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.130456 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-162.ec2.internal" event="NodeReady" Jul 30 20:57:45.130678 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.130602 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 20:57:45.174050 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.174018 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cbq8d"] Jul 30 20:57:45.176637 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.176575 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.177317 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.177294 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dmdcg"] Jul 30 20:57:45.178566 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.178545 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77cz9\"" Jul 30 20:57:45.178699 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.178651 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 20:57:45.178765 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.178724 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 20:57:45.178765 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.178733 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.180372 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.180351 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 20:57:45.180616 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.180598 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 20:57:45.180714 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.180653 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 20:57:45.180991 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.180971 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xbdm8\"" Jul 30 20:57:45.190714 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.190691 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cbq8d"] Jul 30 20:57:45.191372 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.191353 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmdcg"] Jul 30 20:57:45.330401 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.330599 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330415 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt6l6\" (UniqueName: \"kubernetes.io/projected/20a05602-5dbd-4333-a5e1-bc095bb40f1e-kube-api-access-nt6l6\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.330599 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drth6\" (UniqueName: \"kubernetes.io/projected/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-kube-api-access-drth6\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.330599 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330519 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a05602-5dbd-4333-a5e1-bc095bb40f1e-tmp-dir\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.330599 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20a05602-5dbd-4333-a5e1-bc095bb40f1e-config-volume\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.330780 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.330628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-cert\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.431675 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt6l6\" (UniqueName: \"kubernetes.io/projected/20a05602-5dbd-4333-a5e1-bc095bb40f1e-kube-api-access-nt6l6\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.431816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drth6\" (UniqueName: \"kubernetes.io/projected/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-kube-api-access-drth6\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.431816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a05602-5dbd-4333-a5e1-bc095bb40f1e-tmp-dir\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.431816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20a05602-5dbd-4333-a5e1-bc095bb40f1e-config-volume\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.431816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431796 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-cert\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.432014 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.431842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.432014 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:45.431940 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:57:45.432014 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:45.431994 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:57:45.93197641 +0000 UTC m=+36.631112283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:57:45.432179 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.432147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a05602-5dbd-4333-a5e1-bc095bb40f1e-tmp-dir\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.432680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.432658 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/20a05602-5dbd-4333-a5e1-bc095bb40f1e-config-volume\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.436729 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.436572 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-cert\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.442607 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.440041 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt6l6\" (UniqueName: \"kubernetes.io/projected/20a05602-5dbd-4333-a5e1-bc095bb40f1e-kube-api-access-nt6l6\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.442607 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.440383 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drth6\" (UniqueName: \"kubernetes.io/projected/3369fd3e-90aa-4e83-a9c5-b10b171b7d36-kube-api-access-drth6\") pod \"ingress-canary-dmdcg\" (UID: \"3369fd3e-90aa-4e83-a9c5-b10b171b7d36\") " pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.496663 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.496639 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmdcg" Jul 30 20:57:45.756079 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.756045 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:57:45.756499 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.756351 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:57:45.758271 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.758245 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 20:57:45.758399 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.758245 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 20:57:45.758846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.758830 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-66vkk\"" Jul 30 20:57:45.758967 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.758849 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 20:57:45.937556 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:45.936873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:45.937556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:45.937125 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:57:45.937556 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:45.937200 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:57:46.937180549 +0000 UTC m=+37.636316435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:57:46.012767 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.012739 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmdcg"] Jul 30 20:57:46.018406 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:57:46.018379 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3369fd3e_90aa_4e83_a9c5_b10b171b7d36.slice/crio-c07997f7b6dddbaed34aaa271e0e14343196f15f851e1ba4ad526ca445d04716 WatchSource:0}: Error finding container c07997f7b6dddbaed34aaa271e0e14343196f15f851e1ba4ad526ca445d04716: Status 404 returned error can't find the container with id c07997f7b6dddbaed34aaa271e0e14343196f15f851e1ba4ad526ca445d04716 Jul 30 20:57:46.069412 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.069373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmdcg" event={"ID":"3369fd3e-90aa-4e83-a9c5-b10b171b7d36","Type":"ContainerStarted","Data":"c07997f7b6dddbaed34aaa271e0e14343196f15f851e1ba4ad526ca445d04716"} Jul 30 20:57:46.072273 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.072249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerStarted","Data":"825d547588dbeadada17a12e675f1ac4667aa868fd41c5d6629334cdb0c1302a"} Jul 30 20:57:46.752209 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.752181 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:57:46.754215 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.754187 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 20:57:46.754350 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.754270 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5pq\"" Jul 30 20:57:46.944782 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:46.944748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:46.945206 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:46.944913 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:57:46.945206 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:46.944981 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:57:48.944966133 +0000 UTC m=+39.644102001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:57:47.078143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:47.078055 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="825d547588dbeadada17a12e675f1ac4667aa868fd41c5d6629334cdb0c1302a" exitCode=0 Jul 30 20:57:47.078143 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:47.078116 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"825d547588dbeadada17a12e675f1ac4667aa868fd41c5d6629334cdb0c1302a"} Jul 30 20:57:48.085070 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:48.084843 2567 generic.go:356] "Generic (PLEG): container finished" podID="2a37f37e-1a22-459b-ad85-aa6869c43d5c" containerID="6e24e34eac39c6675033c3d73a936b802b56a79d658472f206375512d43f83f0" exitCode=0 Jul 30 20:57:48.085507 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:48.085094 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerDied","Data":"6e24e34eac39c6675033c3d73a936b802b56a79d658472f206375512d43f83f0"} Jul 30 20:57:48.958848 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:48.958825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:48.958987 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:48.958971 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:57:48.959032 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:48.959023 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:57:52.959009341 +0000 UTC m=+43.658145209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:57:49.091004 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:49.090979 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" event={"ID":"2a37f37e-1a22-459b-ad85-aa6869c43d5c","Type":"ContainerStarted","Data":"50948f1d5af2bda90daf1105a7facb6b05f079b9cdd40209f380ada5ff7493ac"} Jul 30 20:57:49.112818 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:49.112773 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zjfqt" podStartSLOduration=9.462310005 podStartE2EDuration="39.112759813s" totalImagesPullingTime="29.650449808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:49.111663131 +0000 UTC m=+39.810799022" watchObservedRunningTime="2026-07-30 20:57:49.112759813 +0000 UTC m=+39.811895703" Jul 30 20:57:50.094465 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:50.094427 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmdcg" event={"ID":"3369fd3e-90aa-4e83-a9c5-b10b171b7d36","Type":"ContainerStarted","Data":"b05c2db7cb564ada588ba5f62c2f57e702dfa4ce6033f4700d34966dc9cd207a"} Jul 30 20:57:50.108487 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:50.108441 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dmdcg" podStartSLOduration=1.9693642489999998 podStartE2EDuration="5.108423394s" totalImagesPullingTime="3.139059145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:50.107450713 +0000 UTC m=+40.806586603" watchObservedRunningTime="2026-07-30 20:57:50.108423394 +0000 UTC m=+40.807559286" Jul 30 20:57:52.982117 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:57:52.982077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:57:52.982477 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:52.982230 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:57:52.982477 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:57:52.982303 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:58:00.982289115 +0000 UTC m=+51.681424982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:58:01.032560 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:01.032521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:01.033051 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:01.032662 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:58:01.033051 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:01.032719 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:58:17.032702967 +0000 UTC m=+67.731838836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:58:05.524197 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.524162 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt"] Jul 30 20:58:05.525966 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.525950 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9"] Jul 30 20:58:05.526086 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.526073 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.527675 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.527654 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.528063 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.528046 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 20:58:05.528836 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.528814 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 20:58:05.528988 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.528966 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 30 20:58:05.529060 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.528990 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 30 20:58:05.529125 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.529061 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r9bw4\"" Jul 30 20:58:05.529630 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.529616 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 30 20:58:05.530781 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.530756 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 30 20:58:05.531209 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.531067 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-znlx7\"" Jul 30 20:58:05.531354 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.531335 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 30 20:58:05.541047 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.541022 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt"] Jul 30 20:58:05.541766 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.541744 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9"] Jul 30 20:58:05.614809 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.614776 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4"] Jul 30 20:58:05.616862 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.616840 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-cpjjc"] Jul 30 20:58:05.616983 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.616969 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" Jul 30 20:58:05.618766 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.618747 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.618898 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.618878 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54fb9b79cd-dhwxl"] Jul 30 20:58:05.619100 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.618912 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r2csq\"" Jul 30 20:58:05.620634 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.620613 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 30 20:58:05.620634 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.620628 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 30 20:58:05.620786 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.620640 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 30 20:58:05.620786 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.620674 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.620786 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.620686 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-k4mkt\"" Jul 30 20:58:05.621059 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.621044 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 30 20:58:05.624666 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.624611 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 20:58:05.624666 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.624621 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 20:58:05.624666 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.624635 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 20:58:05.625051 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.625036 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-klzqk\"" Jul 30 20:58:05.630737 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.630722 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 30 20:58:05.631229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.631213 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 20:58:05.633380 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.633362 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-cpjjc"] Jul 30 20:58:05.634332 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.634312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4"] Jul 30 20:58:05.637205 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.637063 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54fb9b79cd-dhwxl"] Jul 30 20:58:05.665343 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.665319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.665452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.665346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.665452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.665364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvtt8\" (UniqueName: \"kubernetes.io/projected/1931ef5c-2749-415d-a621-3bc663c1d096-kube-api-access-qvtt8\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.665452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.665383 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqm9\" (UniqueName: \"kubernetes.io/projected/673fb888-ff97-4e2c-b34c-287ddb5f5e23-kube-api-access-pvqm9\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.665452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.665408 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1931ef5c-2749-415d-a621-3bc663c1d096-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.766522 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766494 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqm9\" (UniqueName: \"kubernetes.io/projected/673fb888-ff97-4e2c-b34c-287ddb5f5e23-kube-api-access-pvqm9\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766529 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-trusted-ca\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766559 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1931ef5c-2749-415d-a621-3bc663c1d096-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766678 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrxr5\" (UniqueName: \"kubernetes.io/projected/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-kube-api-access-hrxr5\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766702 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766822 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-config\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-serving-cert\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.766955 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766947 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbb9k\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.766981 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.767012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.767080 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.767096 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.767097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvtt8\" (UniqueName: \"kubernetes.io/projected/1931ef5c-2749-415d-a621-3bc663c1d096-kube-api-access-qvtt8\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.767137 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls podName:673fb888-ff97-4e2c-b34c-287ddb5f5e23 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:06.267124102 +0000 UTC m=+56.966259971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-4ccw9" (UID: "673fb888-ff97-4e2c-b34c-287ddb5f5e23") : secret "samples-operator-tls" not found Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.767173 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1931ef5c-2749-415d-a621-3bc663c1d096-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.767189 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:06.26717542 +0000 UTC m=+56.966311305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:05.767256 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.767209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2tg5\" (UniqueName: \"kubernetes.io/projected/99c85ac3-0e43-4430-a61d-b2ff7bd51323-kube-api-access-b2tg5\") pod \"network-check-source-65bdbbff45-wcvn4\" (UID: \"99c85ac3-0e43-4430-a61d-b2ff7bd51323\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" Jul 30 20:58:05.803568 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.803509 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvtt8\" (UniqueName: \"kubernetes.io/projected/1931ef5c-2749-415d-a621-3bc663c1d096-kube-api-access-qvtt8\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:05.803568 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.803544 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqm9\" (UniqueName: \"kubernetes.io/projected/673fb888-ff97-4e2c-b34c-287ddb5f5e23-kube-api-access-pvqm9\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:05.868227 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.868203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-config\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.868227 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.868228 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.868375 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.868246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.868375 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.868263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-serving-cert\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.868853 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.868546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbb9k\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.869084 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869056 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2tg5\" (UniqueName: \"kubernetes.io/projected/99c85ac3-0e43-4430-a61d-b2ff7bd51323-kube-api-access-b2tg5\") pod \"network-check-source-65bdbbff45-wcvn4\" (UID: \"99c85ac3-0e43-4430-a61d-b2ff7bd51323\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" Jul 30 20:58:05.869222 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-trusted-ca\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.869335 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869322 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.869444 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.869552 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.869680 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrxr5\" (UniqueName: \"kubernetes.io/projected/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-kube-api-access-hrxr5\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.869880 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869867 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.869993 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869980 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.870753 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.870724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.871512 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.869547 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:05.871617 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.871521 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:05.871617 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:05.871608 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:06.371571275 +0000 UTC m=+57.070707147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:05.871827 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.869118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-config\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.871882 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.871856 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-trusted-ca\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.872179 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.872155 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.872741 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.872712 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.873102 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.873081 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.874284 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.874265 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-serving-cert\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.874284 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.874275 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.880656 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.880633 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrxr5\" (UniqueName: \"kubernetes.io/projected/6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31-kube-api-access-hrxr5\") pod \"console-operator-5ccd6b445f-cpjjc\" (UID: \"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31\") " pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:05.880749 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.880688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbb9k\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.880803 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.880760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:05.880846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.880809 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2tg5\" (UniqueName: \"kubernetes.io/projected/99c85ac3-0e43-4430-a61d-b2ff7bd51323-kube-api-access-b2tg5\") pod \"network-check-source-65bdbbff45-wcvn4\" (UID: \"99c85ac3-0e43-4430-a61d-b2ff7bd51323\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" Jul 30 20:58:05.926809 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.926778 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" Jul 30 20:58:05.933340 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:05.933314 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:06.053030 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.053003 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4"] Jul 30 20:58:06.056264 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:06.056239 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c85ac3_0e43_4430_a61d_b2ff7bd51323.slice/crio-0b0bb1408c6cdb233df18a83bdd47cdec9b99eaba93220005d3caaff2de40f62 WatchSource:0}: Error finding container 0b0bb1408c6cdb233df18a83bdd47cdec9b99eaba93220005d3caaff2de40f62: Status 404 returned error can't find the container with id 0b0bb1408c6cdb233df18a83bdd47cdec9b99eaba93220005d3caaff2de40f62 Jul 30 20:58:06.067637 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.067615 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-cpjjc"] Jul 30 20:58:06.070230 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:06.070205 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb12ad6_bdc8_41a9_832f_9dcc6df1eb31.slice/crio-f9be5392da3641ab9c168b5702db61f0fcd186ff9e1528dfb535f4d639f2361f WatchSource:0}: Error finding container f9be5392da3641ab9c168b5702db61f0fcd186ff9e1528dfb535f4d639f2361f: Status 404 returned error can't find the container with id f9be5392da3641ab9c168b5702db61f0fcd186ff9e1528dfb535f4d639f2361f Jul 30 20:58:06.130654 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.130628 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerStarted","Data":"f9be5392da3641ab9c168b5702db61f0fcd186ff9e1528dfb535f4d639f2361f"} Jul 30 20:58:06.131679 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.131658 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" event={"ID":"99c85ac3-0e43-4430-a61d-b2ff7bd51323","Type":"ContainerStarted","Data":"0b0bb1408c6cdb233df18a83bdd47cdec9b99eaba93220005d3caaff2de40f62"} Jul 30 20:58:06.273158 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.273118 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:06.273158 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.273162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:06.273356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.273285 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:06.273356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.273289 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 20:58:06.273356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.273335 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:07.273322713 +0000 UTC m=+57.972458581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:06.273356 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.273349 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls podName:673fb888-ff97-4e2c-b34c-287ddb5f5e23 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:07.273342341 +0000 UTC m=+57.972478209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-4ccw9" (UID: "673fb888-ff97-4e2c-b34c-287ddb5f5e23") : secret "samples-operator-tls" not found Jul 30 20:58:06.374264 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:06.374192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:06.374411 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.374360 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:06.374411 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.374377 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:06.374525 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:06.374439 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:07.374420781 +0000 UTC m=+58.073556649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:07.282040 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:07.282002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:07.282496 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:07.282055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:07.282496 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.282182 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 20:58:07.282496 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.282230 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:07.282496 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.282261 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls podName:673fb888-ff97-4e2c-b34c-287ddb5f5e23 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:09.282239286 +0000 UTC m=+59.981375159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-4ccw9" (UID: "673fb888-ff97-4e2c-b34c-287ddb5f5e23") : secret "samples-operator-tls" not found Jul 30 20:58:07.282496 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.282291 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:09.28227406 +0000 UTC m=+59.981409945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:07.383377 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:07.383344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:07.383534 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.383496 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:07.383534 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.383511 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:07.383679 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:07.383562 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:09.383547546 +0000 UTC m=+60.082683426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:08.139127 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:08.139097 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerStarted","Data":"3426c0818e89fe827047f1c1f007e9f390f0e697bfd81cc0bcd915129c029e82"} Jul 30 20:58:08.139344 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:08.139298 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:08.140624 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:08.140598 2567 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-cpjjc container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body= Jul 30 20:58:08.140728 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:08.140642 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused" Jul 30 20:58:08.156526 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:08.156480 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podStartSLOduration=1.252440099 podStartE2EDuration="3.156460264s" totalImagesPullingTime="1.904020165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:08.155509641 +0000 UTC m=+58.854645532" watchObservedRunningTime="2026-07-30 20:58:08.156460264 +0000 UTC m=+58.855596155" Jul 30 20:58:09.142978 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.142954 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/0.log" Jul 30 20:58:09.143347 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.142995 2567 generic.go:356] "Generic (PLEG): container finished" podID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" containerID="3426c0818e89fe827047f1c1f007e9f390f0e697bfd81cc0bcd915129c029e82" exitCode=255 Jul 30 20:58:09.143347 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.143079 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerDied","Data":"3426c0818e89fe827047f1c1f007e9f390f0e697bfd81cc0bcd915129c029e82"} Jul 30 20:58:09.143347 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.143312 2567 scope.go:117] "RemoveContainer" containerID="3426c0818e89fe827047f1c1f007e9f390f0e697bfd81cc0bcd915129c029e82" Jul 30 20:58:09.144658 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.144635 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" event={"ID":"99c85ac3-0e43-4430-a61d-b2ff7bd51323","Type":"ContainerStarted","Data":"ea02ef0fdffb0625bc163a6376dbafa7981a60e3df2f2b0bcf8f96be2c88f2e9"} Jul 30 20:58:09.172067 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.172019 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wcvn4" podStartSLOduration=1.18364357 podStartE2EDuration="4.171999951s" totalImagesPullingTime="2.988356381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:09.171367629 +0000 UTC m=+59.870503520" watchObservedRunningTime="2026-07-30 20:58:09.171999951 +0000 UTC m=+59.871135842" Jul 30 20:58:09.299805 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.299752 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:09.299805 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.299785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:09.299972 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.299916 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 20:58:09.299972 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.299923 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:09.300075 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.299985 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls podName:673fb888-ff97-4e2c-b34c-287ddb5f5e23 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:13.29996436 +0000 UTC m=+63.999100232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-4ccw9" (UID: "673fb888-ff97-4e2c-b34c-287ddb5f5e23") : secret "samples-operator-tls" not found Jul 30 20:58:09.300075 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.300020 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:13.300011213 +0000 UTC m=+63.999147086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:09.401221 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:09.401190 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:09.401358 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.401338 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:09.401419 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.401362 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:09.401457 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:09.401419 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:13.4014008 +0000 UTC m=+64.100536685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:10.065486 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.065457 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sj7bz" Jul 30 20:58:10.148776 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.148750 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/1.log" Jul 30 20:58:10.149237 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.149216 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/0.log" Jul 30 20:58:10.149308 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.149260 2567 generic.go:356] "Generic (PLEG): container finished" podID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" exitCode=255 Jul 30 20:58:10.149376 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.149352 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerDied","Data":"041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8"} Jul 30 20:58:10.149421 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.149409 2567 scope.go:117] "RemoveContainer" containerID="3426c0818e89fe827047f1c1f007e9f390f0e697bfd81cc0bcd915129c029e82" Jul 30 20:58:10.149574 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.149558 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:10.149854 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:10.149825 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:10.816529 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.816496 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g52jz"] Jul 30 20:58:10.845390 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.845359 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g52jz"] Jul 30 20:58:10.845538 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.845480 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:10.847413 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.847391 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 20:58:10.847548 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.847437 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 20:58:10.847548 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.847539 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ws5tw\"" Jul 30 20:58:10.847849 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.847832 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 20:58:10.847937 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.847886 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 20:58:10.913003 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.912965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-crio-socket\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:10.913180 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.913016 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:10.913180 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.913045 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-api-access-p2mzq\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:10.913246 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.913182 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:10.913246 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:10.913204 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-data-volume\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014334 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014334 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014337 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-data-volume\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014507 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-crio-socket\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014507 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014507 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014422 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-api-access-p2mzq\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014507 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.014473 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:11.014732 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014505 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-crio-socket\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.014732 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.014549 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:11.514526994 +0000 UTC m=+62.213662864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:11.014732 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.014553 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 20:58:11.014732 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.014640 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:11.514623462 +0000 UTC m=+62.213759337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : configmap "kube-rbac-proxy" not found Jul 30 20:58:11.014941 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.014772 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-data-volume\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.022227 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.022196 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2mzq\" (UniqueName: \"kubernetes.io/projected/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-api-access-p2mzq\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.152839 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.152771 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/1.log" Jul 30 20:58:11.153287 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.153179 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:11.153381 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.153355 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:11.519203 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.519164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.519433 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.519286 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 20:58:11.519433 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:11.519295 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:11.519433 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.519352 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:12.519332795 +0000 UTC m=+63.218468686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : configmap "kube-rbac-proxy" not found Jul 30 20:58:11.519433 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.519402 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:11.519570 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:11.519436 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:12.519426008 +0000 UTC m=+63.218561882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:12.233517 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.233487 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz"] Jul 30 20:58:12.236230 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.236209 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" Jul 30 20:58:12.238417 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.238400 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 30 20:58:12.238954 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.238937 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8lcmf\"" Jul 30 20:58:12.239053 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.238989 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 30 20:58:12.254712 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.254688 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz"] Jul 30 20:58:12.327569 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.327541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59bmz\" (UniqueName: \"kubernetes.io/projected/c03ee922-4a2a-4ffb-a540-e0bec4d4e77c-kube-api-access-59bmz\") pod \"migrator-68bc745d87-jp4mz\" (UID: \"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" Jul 30 20:58:12.428035 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.427997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59bmz\" (UniqueName: \"kubernetes.io/projected/c03ee922-4a2a-4ffb-a540-e0bec4d4e77c-kube-api-access-59bmz\") pod \"migrator-68bc745d87-jp4mz\" (UID: \"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" Jul 30 20:58:12.436255 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.436232 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59bmz\" (UniqueName: \"kubernetes.io/projected/c03ee922-4a2a-4ffb-a540-e0bec4d4e77c-kube-api-access-59bmz\") pod \"migrator-68bc745d87-jp4mz\" (UID: \"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" Jul 30 20:58:12.529230 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.529195 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:12.529399 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.529288 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:12.529399 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:12.529343 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:12.529399 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:12.529378 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 20:58:12.529524 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:12.529408 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:14.529392634 +0000 UTC m=+65.228528502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:12.529524 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:12.529428 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:14.529413719 +0000 UTC m=+65.228549602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : configmap "kube-rbac-proxy" not found Jul 30 20:58:12.544722 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.544702 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" Jul 30 20:58:12.567392 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.567371 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-45ms9_8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4/dns-node-resolver/0.log" Jul 30 20:58:12.658525 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:12.658497 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz"] Jul 30 20:58:12.661556 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:12.661527 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03ee922_4a2a_4ffb_a540_e0bec4d4e77c.slice/crio-72dcb470bd415352b7d6de33043b6eabe1755e8417189d06ebaa7a4cf9b10da1 WatchSource:0}: Error finding container 72dcb470bd415352b7d6de33043b6eabe1755e8417189d06ebaa7a4cf9b10da1: Status 404 returned error can't find the container with id 72dcb470bd415352b7d6de33043b6eabe1755e8417189d06ebaa7a4cf9b10da1 Jul 30 20:58:13.158500 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:13.158465 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" event={"ID":"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c","Type":"ContainerStarted","Data":"72dcb470bd415352b7d6de33043b6eabe1755e8417189d06ebaa7a4cf9b10da1"} Jul 30 20:58:13.335807 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:13.335771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:13.336246 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:13.335894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:13.336246 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.335933 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:13.336246 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.336011 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:21.33599215 +0000 UTC m=+72.035128019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:13.336246 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.336059 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 20:58:13.336246 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.336125 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls podName:673fb888-ff97-4e2c-b34c-287ddb5f5e23 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:21.336109949 +0000 UTC m=+72.035245818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-4ccw9" (UID: "673fb888-ff97-4e2c-b34c-287ddb5f5e23") : secret "samples-operator-tls" not found Jul 30 20:58:13.436854 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:13.436770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:13.437007 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.436923 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:13.437007 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.436942 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:13.437007 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:13.437004 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:21.436982959 +0000 UTC m=+72.136118877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:13.767384 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:13.767355 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nk9g2_597f0712-04ff-4ba3-b59d-5df371cd7a8f/node-ca/0.log" Jul 30 20:58:14.008279 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.008249 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-gdvf9"] Jul 30 20:58:14.010268 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.010252 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.012266 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.012242 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 30 20:58:14.012780 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.012758 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 30 20:58:14.012885 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.012759 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-prwpr\"" Jul 30 20:58:14.012885 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.012796 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 30 20:58:14.012885 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.012764 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 30 20:58:14.018253 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.018206 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-gdvf9"] Jul 30 20:58:14.042144 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.042120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-cabundle\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.042240 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.042167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-key\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.042304 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.042240 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmf67\" (UniqueName: \"kubernetes.io/projected/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-kube-api-access-mmf67\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.142926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.142887 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-cabundle\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.143073 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.142951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-key\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.143073 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.142988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmf67\" (UniqueName: \"kubernetes.io/projected/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-kube-api-access-mmf67\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.143560 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.143535 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-cabundle\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.145294 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.145269 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-signing-key\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.151282 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.151260 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmf67\" (UniqueName: \"kubernetes.io/projected/c67980eb-2301-41fd-88fa-2fb50e7e3c6d-kube-api-access-mmf67\") pod \"service-ca-98c88fbb5-gdvf9\" (UID: \"c67980eb-2301-41fd-88fa-2fb50e7e3c6d\") " pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.162733 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.162707 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" event={"ID":"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c","Type":"ContainerStarted","Data":"8e5f9c05307dd36989966d0932df57d4cb4b03420760116b514fe6fdc1d031ec"} Jul 30 20:58:14.162826 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.162758 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" event={"ID":"c03ee922-4a2a-4ffb-a540-e0bec4d4e77c","Type":"ContainerStarted","Data":"3868f7713206d446a5b801f884ffb5da431bcda6252cff07fa75f8b2f0eedce1"} Jul 30 20:58:14.177707 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.177663 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-jp4mz" podStartSLOduration=1.008182423 podStartE2EDuration="2.177650112s" totalImagesPullingTime="1.169467689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:14.176673452 +0000 UTC m=+64.875809354" watchObservedRunningTime="2026-07-30 20:58:14.177650112 +0000 UTC m=+64.876786054" Jul 30 20:58:14.319783 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.319713 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" Jul 30 20:58:14.369944 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.369916 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmdcg_3369fd3e-90aa-4e83-a9c5-b10b171b7d36/serve-healthcheck-canary/0.log" Jul 30 20:58:14.434777 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.434623 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-gdvf9"] Jul 30 20:58:14.437033 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:14.437005 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67980eb_2301_41fd_88fa_2fb50e7e3c6d.slice/crio-493fd471fc5a46e5a33611ec6d9bb38d892c5034d50e9103abe6229054042781 WatchSource:0}: Error finding container 493fd471fc5a46e5a33611ec6d9bb38d892c5034d50e9103abe6229054042781: Status 404 returned error can't find the container with id 493fd471fc5a46e5a33611ec6d9bb38d892c5034d50e9103abe6229054042781 Jul 30 20:58:14.546904 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.546872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:14.547083 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:14.546967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:14.547083 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:14.547017 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:14.547174 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:14.547081 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 20:58:14.547174 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:14.547083 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:18.547067051 +0000 UTC m=+69.246202918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:14.547174 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:14.547139 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:18.54712426 +0000 UTC m=+69.246260131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : configmap "kube-rbac-proxy" not found Jul 30 20:58:15.165988 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.165952 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" event={"ID":"c67980eb-2301-41fd-88fa-2fb50e7e3c6d","Type":"ContainerStarted","Data":"493fd471fc5a46e5a33611ec6d9bb38d892c5034d50e9103abe6229054042781"} Jul 30 20:58:15.455836 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.455759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:58:15.458596 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.458563 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 20:58:15.466359 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:15.466333 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 20:58:15.466465 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:15.466411 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs podName:f8ad8626-c218-4c3f-918d-4295881c08e3 nodeName:}" failed. No retries permitted until 2026-07-30 20:59:19.466391931 +0000 UTC m=+130.165527816 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs") pod "network-metrics-daemon-qct22" (UID: "f8ad8626-c218-4c3f-918d-4295881c08e3") : secret "metrics-daemon-secret" not found Jul 30 20:58:15.556655 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.556618 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:58:15.556813 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.556724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:58:15.559106 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.559081 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 20:58:15.559915 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.559889 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqrq\" (UniqueName: \"kubernetes.io/projected/3b462055-f414-4a97-ac4c-9a8e02a889e0-kube-api-access-dmqrq\") pod \"network-check-target-w5rlj\" (UID: \"3b462055-f414-4a97-ac4c-9a8e02a889e0\") " pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:58:15.569907 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.569885 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/53886e58-f91a-4e1f-a5a2-331f73bc8668-original-pull-secret\") pod \"global-pull-secret-syncer-ptckn\" (UID: \"53886e58-f91a-4e1f-a5a2-331f73bc8668\") " pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:58:15.770501 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.770472 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-66vkk\"" Jul 30 20:58:15.778108 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.778088 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptckn" Jul 30 20:58:15.779162 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.779138 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:58:15.925021 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.924985 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ptckn"] Jul 30 20:58:15.928360 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:15.928328 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53886e58_f91a_4e1f_a5a2_331f73bc8668.slice/crio-0c20c1671bb59415d7ef3e70129fef4b35fc6cef851c2faac7ce1cf9cea76541 WatchSource:0}: Error finding container 0c20c1671bb59415d7ef3e70129fef4b35fc6cef851c2faac7ce1cf9cea76541: Status 404 returned error can't find the container with id 0c20c1671bb59415d7ef3e70129fef4b35fc6cef851c2faac7ce1cf9cea76541 Jul 30 20:58:15.934130 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.934109 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:15.934521 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.934504 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:15.934720 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:15.934686 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:15.937389 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:15.937324 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w5rlj"] Jul 30 20:58:15.939817 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:15.939797 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b462055_f414_4a97_ac4c_9a8e02a889e0.slice/crio-7ded78e64fdceb65dc59366294a66b19d87ed9bef5e15ad54042847097f1a4c2 WatchSource:0}: Error finding container 7ded78e64fdceb65dc59366294a66b19d87ed9bef5e15ad54042847097f1a4c2: Status 404 returned error can't find the container with id 7ded78e64fdceb65dc59366294a66b19d87ed9bef5e15ad54042847097f1a4c2 Jul 30 20:58:16.170301 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:16.170209 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w5rlj" event={"ID":"3b462055-f414-4a97-ac4c-9a8e02a889e0","Type":"ContainerStarted","Data":"7ded78e64fdceb65dc59366294a66b19d87ed9bef5e15ad54042847097f1a4c2"} Jul 30 20:58:16.171433 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:16.171406 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ptckn" event={"ID":"53886e58-f91a-4e1f-a5a2-331f73bc8668","Type":"ContainerStarted","Data":"0c20c1671bb59415d7ef3e70129fef4b35fc6cef851c2faac7ce1cf9cea76541"} Jul 30 20:58:17.069744 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.069707 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:17.070208 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:17.070178 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 20:58:17.070297 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:17.070274 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls podName:20a05602-5dbd-4333-a5e1-bc095bb40f1e nodeName:}" failed. No retries permitted until 2026-07-30 20:58:49.070245472 +0000 UTC m=+99.769381340 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls") pod "dns-default-cbq8d" (UID: "20a05602-5dbd-4333-a5e1-bc095bb40f1e") : secret "dns-default-metrics-tls" not found Jul 30 20:58:17.176627 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.176591 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w5rlj" event={"ID":"3b462055-f414-4a97-ac4c-9a8e02a889e0","Type":"ContainerStarted","Data":"2a29a9f85a6bd64c147a11435be2952b4852041702e24549271453613d71a3b5"} Jul 30 20:58:17.176786 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.176694 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:58:17.178657 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.178575 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" event={"ID":"c67980eb-2301-41fd-88fa-2fb50e7e3c6d","Type":"ContainerStarted","Data":"ab62a00d5d4abe3029e4484cb52fd723e1ab2c0fd592566aa9cc485e6c18ac4c"} Jul 30 20:58:17.192051 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.192003 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w5rlj" podStartSLOduration=67.191986804 podStartE2EDuration="1m7.191986804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:17.191147684 +0000 UTC m=+67.890283575" watchObservedRunningTime="2026-07-30 20:58:17.191986804 +0000 UTC m=+67.891122674" Jul 30 20:58:17.205804 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:17.205761 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-gdvf9" podStartSLOduration=2.164506811 podStartE2EDuration="4.205749335s" totalImagesPullingTime="2.041242524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:17.204742519 +0000 UTC m=+67.903878411" watchObservedRunningTime="2026-07-30 20:58:17.205749335 +0000 UTC m=+67.904885225" Jul 30 20:58:18.139595 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:18.139556 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:18.140049 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:18.140032 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:18.140252 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:18.140226 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:18.581503 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:18.581449 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:18.581688 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:18.581569 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:18.581688 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:18.581592 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 20:58:18.581688 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:18.581662 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:26.581641341 +0000 UTC m=+77.280777210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : configmap "kube-rbac-proxy" not found Jul 30 20:58:18.581813 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:18.581724 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:18.581813 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:18.581770 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:26.581757735 +0000 UTC m=+77.280893610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:21.194066 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.194026 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ptckn" event={"ID":"53886e58-f91a-4e1f-a5a2-331f73bc8668","Type":"ContainerStarted","Data":"a6ea7d4b3d002215a651272814b52eaf5a43a1b7033f3d83f7bd60081f49eec2"} Jul 30 20:58:21.409016 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.408977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:21.409211 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.409051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:21.409211 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:21.409129 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:21.409211 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:21.409204 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls podName:1931ef5c-2749-415d-a621-3bc663c1d096 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:37.409185071 +0000 UTC m=+88.108320943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-rbnpt" (UID: "1931ef5c-2749-415d-a621-3bc663c1d096") : secret "cluster-monitoring-operator-tls" not found Jul 30 20:58:21.411341 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.411322 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/673fb888-ff97-4e2c-b34c-287ddb5f5e23-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-4ccw9\" (UID: \"673fb888-ff97-4e2c-b34c-287ddb5f5e23\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:21.445652 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.445602 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-znlx7\"" Jul 30 20:58:21.454211 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.454194 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" Jul 30 20:58:21.510358 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.510058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") pod \"image-registry-54fb9b79cd-dhwxl\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:21.510358 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:21.510302 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 20:58:21.510358 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:21.510317 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fb9b79cd-dhwxl: secret "image-registry-tls" not found Jul 30 20:58:21.510611 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:21.510382 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls podName:8c2f3a9a-83d2-4079-9671-d81840392995 nodeName:}" failed. No retries permitted until 2026-07-30 20:58:37.510362387 +0000 UTC m=+88.209498268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls") pod "image-registry-54fb9b79cd-dhwxl" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995") : secret "image-registry-tls" not found Jul 30 20:58:21.603115 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.603058 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ptckn" podStartSLOduration=67.414266814 podStartE2EDuration="1m11.60303515s" totalImagesPullingTime="4.188768336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:21.208154161 +0000 UTC m=+71.907290051" watchObservedRunningTime="2026-07-30 20:58:21.60303515 +0000 UTC m=+72.302171041" Jul 30 20:58:21.604138 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:21.604119 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9"] Jul 30 20:58:22.199232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:22.199195 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" event={"ID":"673fb888-ff97-4e2c-b34c-287ddb5f5e23","Type":"ContainerStarted","Data":"6c38aa1428a138bfebc875d07e67549c7bdd0e0809e609ca9e9a3cf1f664f3a2"} Jul 30 20:58:24.210341 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:24.210306 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" event={"ID":"673fb888-ff97-4e2c-b34c-287ddb5f5e23","Type":"ContainerStarted","Data":"ea4316902e70c3d4df32825535ac630a5dac80a5bedf9f35c2994fe777ede000"} Jul 30 20:58:24.210827 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:24.210401 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" event={"ID":"673fb888-ff97-4e2c-b34c-287ddb5f5e23","Type":"ContainerStarted","Data":"397e6d5fc8dd38870ab9d92cc814b142607a9b11f227ba9c5761944aeb02923d"} Jul 30 20:58:24.229007 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:24.228957 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-4ccw9" podStartSLOduration=17.47179671 podStartE2EDuration="19.228945112s" totalImagesPullingTime="1.757148402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:24.228113279 +0000 UTC m=+74.927249168" watchObservedRunningTime="2026-07-30 20:58:24.228945112 +0000 UTC m=+74.928081001" Jul 30 20:58:26.652433 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:26.652396 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:26.652854 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:26.652460 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:26.652854 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:26.652552 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 20:58:26.652854 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:26.652654 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls podName:20ef0cf3-01e8-4b97-a573-f3ee67b25dbe nodeName:}" failed. No retries permitted until 2026-07-30 20:58:42.652634428 +0000 UTC m=+93.351770301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g52jz" (UID: "20ef0cf3-01e8-4b97-a573-f3ee67b25dbe") : secret "insights-runtime-extractor-tls" not found Jul 30 20:58:26.653101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:26.653083 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:30.752369 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:30.752330 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:31.232431 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.232362 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 20:58:31.232768 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.232755 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/1.log" Jul 30 20:58:31.232834 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.232787 2567 generic.go:356] "Generic (PLEG): container finished" podID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" exitCode=255 Jul 30 20:58:31.232870 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.232859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerDied","Data":"5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd"} Jul 30 20:58:31.232921 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.232908 2567 scope.go:117] "RemoveContainer" containerID="041a85a6e6e73e0b405e29ead499ea84eee902f172a12baa72bd9cd73209efc8" Jul 30 20:58:31.233216 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:31.233190 2567 scope.go:117] "RemoveContainer" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" Jul 30 20:58:31.233405 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:31.233382 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:32.237430 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:32.237327 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 20:58:34.706080 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:34.706046 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54fb9b79cd-dhwxl"] Jul 30 20:58:34.706986 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:34.706954 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" podUID="8c2f3a9a-83d2-4079-9671-d81840392995" Jul 30 20:58:35.246165 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.246138 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:35.251898 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.251876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:35.315998 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.315971 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316132 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316029 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316132 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316055 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbb9k\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316132 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316092 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316132 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316121 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316328 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316151 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316328 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316181 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets\") pod \"8c2f3a9a-83d2-4079-9671-d81840392995\" (UID: \"8c2f3a9a-83d2-4079-9671-d81840392995\") " Jul 30 20:58:35.316493 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316444 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 20:58:35.316493 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316450 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 20:58:35.316694 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.316501 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 20:58:35.318379 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.318350 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 20:58:35.318473 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.318371 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k" (OuterVolumeSpecName: "kube-api-access-tbb9k") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "kube-api-access-tbb9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 20:58:35.318621 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.318601 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 20:58:35.318686 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.318622 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8c2f3a9a-83d2-4079-9671-d81840392995" (UID: "8c2f3a9a-83d2-4079-9671-d81840392995"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 20:58:35.417161 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417127 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-trusted-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417161 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417153 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbb9k\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-kube-api-access-tbb9k\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417161 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417164 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-image-registry-private-configuration\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417377 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417181 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c2f3a9a-83d2-4079-9671-d81840392995-registry-certificates\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417377 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417190 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c2f3a9a-83d2-4079-9671-d81840392995-ca-trust-extracted\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417377 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417198 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c2f3a9a-83d2-4079-9671-d81840392995-installation-pull-secrets\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.417377 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.417207 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-bound-sa-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:35.933775 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.933744 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:35.934141 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:35.934016 2567 scope.go:117] "RemoveContainer" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" Jul 30 20:58:35.934182 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:35.934156 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:36.249227 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:36.249205 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fb9b79cd-dhwxl" Jul 30 20:58:36.275754 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:36.275728 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54fb9b79cd-dhwxl"] Jul 30 20:58:36.279270 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:36.279249 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54fb9b79cd-dhwxl"] Jul 30 20:58:36.322719 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:36.322694 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c2f3a9a-83d2-4079-9671-d81840392995-registry-tls\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 20:58:37.430161 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.430130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:37.432391 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.432371 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1931ef5c-2749-415d-a621-3bc663c1d096-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-rbnpt\" (UID: \"1931ef5c-2749-415d-a621-3bc663c1d096\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:37.640443 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.640421 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r9bw4\"" Jul 30 20:58:37.648927 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.648909 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" Jul 30 20:58:37.755896 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.755869 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c2f3a9a-83d2-4079-9671-d81840392995" path="/var/lib/kubelet/pods/8c2f3a9a-83d2-4079-9671-d81840392995/volumes" Jul 30 20:58:37.773366 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:37.773340 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt"] Jul 30 20:58:37.776646 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:37.776618 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1931ef5c_2749_415d_a621_3bc663c1d096.slice/crio-93a65086cb2581fa2e79ecb46504a38a558f55b087cd090b130a7b985de3b501 WatchSource:0}: Error finding container 93a65086cb2581fa2e79ecb46504a38a558f55b087cd090b130a7b985de3b501: Status 404 returned error can't find the container with id 93a65086cb2581fa2e79ecb46504a38a558f55b087cd090b130a7b985de3b501 Jul 30 20:58:38.140126 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:38.140101 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:58:38.140421 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:38.140409 2567 scope.go:117] "RemoveContainer" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" Jul 30 20:58:38.140559 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:38.140540 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:38.258728 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:38.258693 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" event={"ID":"1931ef5c-2749-415d-a621-3bc663c1d096","Type":"ContainerStarted","Data":"93a65086cb2581fa2e79ecb46504a38a558f55b087cd090b130a7b985de3b501"} Jul 30 20:58:40.266408 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:40.266369 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" event={"ID":"1931ef5c-2749-415d-a621-3bc663c1d096","Type":"ContainerStarted","Data":"a5c0e8b651baa524e5deb05f3c803e69e947ab3bc2904c12aac2d02b56932867"} Jul 30 20:58:40.283180 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:40.283133 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-rbnpt" podStartSLOduration=33.542829738 podStartE2EDuration="35.283119854s" totalImagesPullingTime="1.740290116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:40.282553047 +0000 UTC m=+90.981688936" watchObservedRunningTime="2026-07-30 20:58:40.283119854 +0000 UTC m=+90.982255748" Jul 30 20:58:42.067749 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.067711 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-c764m"] Jul 30 20:58:42.100929 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.100898 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-c764m"] Jul 30 20:58:42.101069 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.100988 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.103528 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.103506 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 30 20:58:42.103724 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.103675 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 20:58:42.103785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.103723 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 30 20:58:42.103785 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.103753 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rqzf7\"" Jul 30 20:58:42.168495 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.168464 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f61fbd71-93d9-4f66-9c27-11cda64a44fd-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.168635 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.168593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.168635 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.168626 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9z9v\" (UniqueName: \"kubernetes.io/projected/f61fbd71-93d9-4f66-9c27-11cda64a44fd-kube-api-access-q9z9v\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.168714 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.168700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.269020 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.268985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.269020 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.269019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9z9v\" (UniqueName: \"kubernetes.io/projected/f61fbd71-93d9-4f66-9c27-11cda64a44fd-kube-api-access-q9z9v\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.269207 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.269057 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.269207 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.269092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f61fbd71-93d9-4f66-9c27-11cda64a44fd-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.270289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.270262 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f61fbd71-93d9-4f66-9c27-11cda64a44fd-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.274341 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.272415 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.275156 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.275135 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f61fbd71-93d9-4f66-9c27-11cda64a44fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.277353 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.277332 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9z9v\" (UniqueName: \"kubernetes.io/projected/f61fbd71-93d9-4f66-9c27-11cda64a44fd-kube-api-access-q9z9v\") pod \"prometheus-operator-758d7cb874-c764m\" (UID: \"f61fbd71-93d9-4f66-9c27-11cda64a44fd\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.411699 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.411612 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" Jul 30 20:58:42.534147 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.534116 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-c764m"] Jul 30 20:58:42.537153 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:42.537126 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61fbd71_93d9_4f66_9c27_11cda64a44fd.slice/crio-b53a189b62d11565c0d4bf90a45d7561f66b09a2f4ec44188e43140e85a2d9ff WatchSource:0}: Error finding container b53a189b62d11565c0d4bf90a45d7561f66b09a2f4ec44188e43140e85a2d9ff: Status 404 returned error can't find the container with id b53a189b62d11565c0d4bf90a45d7561f66b09a2f4ec44188e43140e85a2d9ff Jul 30 20:58:42.672708 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.672644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:42.674737 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.674709 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/20ef0cf3-01e8-4b97-a573-f3ee67b25dbe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g52jz\" (UID: \"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe\") " pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:42.954217 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:42.954126 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g52jz" Jul 30 20:58:43.074220 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:43.074186 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g52jz"] Jul 30 20:58:43.077842 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:43.077811 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ef0cf3_01e8_4b97_a573_f3ee67b25dbe.slice/crio-256e69f5f67de0d02c140c9999a62336a01e283a2a3affb14ae017c658b6f39b WatchSource:0}: Error finding container 256e69f5f67de0d02c140c9999a62336a01e283a2a3affb14ae017c658b6f39b: Status 404 returned error can't find the container with id 256e69f5f67de0d02c140c9999a62336a01e283a2a3affb14ae017c658b6f39b Jul 30 20:58:43.279756 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:43.279681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" event={"ID":"f61fbd71-93d9-4f66-9c27-11cda64a44fd","Type":"ContainerStarted","Data":"b53a189b62d11565c0d4bf90a45d7561f66b09a2f4ec44188e43140e85a2d9ff"} Jul 30 20:58:43.281868 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:43.281824 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g52jz" event={"ID":"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe","Type":"ContainerStarted","Data":"8dae0104a346fd5b43a829406c387c41e87a3bd16625eaa769d45ea869f370fa"} Jul 30 20:58:43.282016 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:43.281889 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g52jz" event={"ID":"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe","Type":"ContainerStarted","Data":"256e69f5f67de0d02c140c9999a62336a01e283a2a3affb14ae017c658b6f39b"} Jul 30 20:58:44.287573 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:44.287532 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" event={"ID":"f61fbd71-93d9-4f66-9c27-11cda64a44fd","Type":"ContainerStarted","Data":"98e1c9f8e8d806c32e535f7633c2bf33b7bde7f5b9e45778accc72ec96db668c"} Jul 30 20:58:44.288030 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:44.287627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" event={"ID":"f61fbd71-93d9-4f66-9c27-11cda64a44fd","Type":"ContainerStarted","Data":"4221b5959178e727392bf6047e9fa7302124e7e153203b4cf3f66df50e2ede72"} Jul 30 20:58:44.289396 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:44.289359 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g52jz" event={"ID":"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe","Type":"ContainerStarted","Data":"e25cf6990a4e700216d8ae2a3b5bf87ed906eb362cc0115aee7a274b3f1b2709"} Jul 30 20:58:44.304879 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:44.304818 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-c764m" podStartSLOduration=0.782491315 podStartE2EDuration="2.304796777s" totalImagesPullingTime="1.522305462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:44.304303202 +0000 UTC m=+95.003439093" watchObservedRunningTime="2026-07-30 20:58:44.304796777 +0000 UTC m=+95.003932671" Jul 30 20:58:45.294708 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:45.294678 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g52jz" event={"ID":"20ef0cf3-01e8-4b97-a573-f3ee67b25dbe","Type":"ContainerStarted","Data":"816ebded70747e29ffc8834d4353c7ced3731c348ae84ea72b10371b50d31325"} Jul 30 20:58:45.314502 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:45.314455 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g52jz" podStartSLOduration=33.392980682 podStartE2EDuration="35.314437848s" totalImagesPullingTime="1.921457166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:58:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:45.313896796 +0000 UTC m=+96.013032684" watchObservedRunningTime="2026-07-30 20:58:45.314437848 +0000 UTC m=+96.013573739" Jul 30 20:58:46.461459 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.461425 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66"] Jul 30 20:58:46.464758 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.464737 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.466703 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.466684 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 30 20:58:46.467519 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.467488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 30 20:58:46.467519 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.467507 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 30 20:58:46.467707 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.467494 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-v7gqp\"" Jul 30 20:58:46.480379 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.480359 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66"] Jul 30 20:58:46.493032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.493010 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lnl62"] Jul 30 20:58:46.496432 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.496417 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.498446 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.498214 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 20:58:46.498446 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.498242 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 20:58:46.498621 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.498518 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 20:58:46.498621 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.498556 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6shkw\"" Jul 30 20:58:46.603701 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-wtmp\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwnrw\" (UniqueName: \"kubernetes.io/projected/ddb0ab00-236d-45e1-896a-18270539a50b-kube-api-access-qwnrw\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603816 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-textfile\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603814 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-root\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603849 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-sys\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603868 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.603926 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603922 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e463673-a152-4063-adba-af75c5e1b51a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.603944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.604050 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-metrics-client-ca\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.604098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.604135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.604159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.604232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.604185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj59c\" (UniqueName: \"kubernetes.io/projected/9e463673-a152-4063-adba-af75c5e1b51a-kube-api-access-hj59c\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.704665 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.704833 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj59c\" (UniqueName: \"kubernetes.io/projected/9e463673-a152-4063-adba-af75c5e1b51a-kube-api-access-hj59c\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.704833 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704708 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-wtmp\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.704833 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwnrw\" (UniqueName: \"kubernetes.io/projected/ddb0ab00-236d-45e1-896a-18270539a50b-kube-api-access-qwnrw\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.704833 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-textfile\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705037 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704921 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-root\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705037 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704922 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-wtmp\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705037 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704981 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-sys\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705037 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.704957 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-root\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705037 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705046 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb0ab00-236d-45e1-896a-18270539a50b-sys\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705039 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705082 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705134 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-textfile\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705118 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e463673-a152-4063-adba-af75c5e1b51a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705175 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705253 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-metrics-client-ca\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705289 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:46.705273 2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 30 20:58:46.705704 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.705704 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:46.705339 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls podName:ddb0ab00-236d-45e1-896a-18270539a50b nodeName:}" failed. No retries permitted until 2026-07-30 20:58:47.205319818 +0000 UTC m=+97.904455700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls") pod "node-exporter-lnl62" (UID: "ddb0ab00-236d-45e1-896a-18270539a50b") : secret "node-exporter-tls" not found Jul 30 20:58:46.705704 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705373 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.705704 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705680 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.705912 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.705845 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb0ab00-236d-45e1-896a-18270539a50b-metrics-client-ca\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.706139 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.706117 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e463673-a152-4063-adba-af75c5e1b51a-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.706281 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.706261 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.706537 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.706513 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.707304 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.707286 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.707530 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.707501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.707909 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.707888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e463673-a152-4063-adba-af75c5e1b51a-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.712494 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.712452 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwnrw\" (UniqueName: \"kubernetes.io/projected/ddb0ab00-236d-45e1-896a-18270539a50b-kube-api-access-qwnrw\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:46.715686 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.715664 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj59c\" (UniqueName: \"kubernetes.io/projected/9e463673-a152-4063-adba-af75c5e1b51a-kube-api-access-hj59c\") pod \"kube-state-metrics-6bd9664c4-nqg66\" (UID: \"9e463673-a152-4063-adba-af75c5e1b51a\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.774083 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.774007 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" Jul 30 20:58:46.903522 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:46.903493 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66"] Jul 30 20:58:46.907917 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:46.907878 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e463673_a152_4063_adba_af75c5e1b51a.slice/crio-19245f78bdca879ba0d77bcbd201fbe1507dd6f5499ccf4a3bed2c5bda9ab963 WatchSource:0}: Error finding container 19245f78bdca879ba0d77bcbd201fbe1507dd6f5499ccf4a3bed2c5bda9ab963: Status 404 returned error can't find the container with id 19245f78bdca879ba0d77bcbd201fbe1507dd6f5499ccf4a3bed2c5bda9ab963 Jul 30 20:58:47.209695 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:47.209657 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:47.211824 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:47.211808 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb0ab00-236d-45e1-896a-18270539a50b-node-exporter-tls\") pod \"node-exporter-lnl62\" (UID: \"ddb0ab00-236d-45e1-896a-18270539a50b\") " pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:47.301778 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:47.301740 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" event={"ID":"9e463673-a152-4063-adba-af75c5e1b51a","Type":"ContainerStarted","Data":"19245f78bdca879ba0d77bcbd201fbe1507dd6f5499ccf4a3bed2c5bda9ab963"} Jul 30 20:58:47.408255 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:47.408223 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lnl62" Jul 30 20:58:47.420510 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:47.420484 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb0ab00_236d_45e1_896a_18270539a50b.slice/crio-84123f16399c388f006778f2934ac7cef50c06d3c452ad600a81bfb289a3e664 WatchSource:0}: Error finding container 84123f16399c388f006778f2934ac7cef50c06d3c452ad600a81bfb289a3e664: Status 404 returned error can't find the container with id 84123f16399c388f006778f2934ac7cef50c06d3c452ad600a81bfb289a3e664 Jul 30 20:58:48.184233 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.184203 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w5rlj" Jul 30 20:58:48.307560 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.307524 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnl62" event={"ID":"ddb0ab00-236d-45e1-896a-18270539a50b","Type":"ContainerStarted","Data":"84123f16399c388f006778f2934ac7cef50c06d3c452ad600a81bfb289a3e664"} Jul 30 20:58:48.421293 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.421260 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-598977cc7d-hfgvx"] Jul 30 20:58:48.426707 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.426681 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.429859 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.429476 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 30 20:58:48.429859 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.429511 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 30 20:58:48.429859 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.429531 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 30 20:58:48.429859 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.429536 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dc9jutke9ievt\"" Jul 30 20:58:48.429859 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.429512 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 30 20:58:48.430302 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.430274 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 30 20:58:48.430438 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.430341 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xsf6d\"" Jul 30 20:58:48.437060 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.437039 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-598977cc7d-hfgvx"] Jul 30 20:58:48.519653 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519621 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-grpc-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.519870 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-metrics-client-ca\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.519870 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.519870 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519859 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.520042 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519895 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.520042 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519925 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.520042 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519953 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkvml\" (UniqueName: \"kubernetes.io/projected/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-kube-api-access-zkvml\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.520042 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.519995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621047 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621017 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkvml\" (UniqueName: \"kubernetes.io/projected/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-kube-api-access-zkvml\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621165 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621229 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621221 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-grpc-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621265 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-metrics-client-ca\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.621452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.621293 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.622265 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.622234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-metrics-client-ca\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.624180 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.624151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.624902 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.624689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.624902 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.624693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.624902 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.624851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-grpc-tls\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.625211 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.625174 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.625500 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.625475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.629783 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.629763 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkvml\" (UniqueName: \"kubernetes.io/projected/6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b-kube-api-access-zkvml\") pod \"thanos-querier-598977cc7d-hfgvx\" (UID: \"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b\") " pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.738152 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.738082 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:48.954306 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:48.954193 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-598977cc7d-hfgvx"] Jul 30 20:58:48.957890 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:48.957723 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe4b2a6_3641_4f4f_bb7a_5d5ac826366b.slice/crio-23a94640761bd1e98e6663d658d65c370e7ba19974ff4451eb5fc2bf90c28844 WatchSource:0}: Error finding container 23a94640761bd1e98e6663d658d65c370e7ba19974ff4451eb5fc2bf90c28844: Status 404 returned error can't find the container with id 23a94640761bd1e98e6663d658d65c370e7ba19974ff4451eb5fc2bf90c28844 Jul 30 20:58:49.125666 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.125630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:49.127767 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.127748 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/20a05602-5dbd-4333-a5e1-bc095bb40f1e-metrics-tls\") pod \"dns-default-cbq8d\" (UID: \"20a05602-5dbd-4333-a5e1-bc095bb40f1e\") " pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:49.313778 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.313683 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" event={"ID":"9e463673-a152-4063-adba-af75c5e1b51a","Type":"ContainerStarted","Data":"1ee85445361708875523a0dd60c290cf1ff026b37c8bfde730fb8a9ad569ead4"} Jul 30 20:58:49.313778 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.313752 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" event={"ID":"9e463673-a152-4063-adba-af75c5e1b51a","Type":"ContainerStarted","Data":"4eda00583d6e3941a8fa271b432700e534d3d105e4d3f49bd4f54f5a52f93f0e"} Jul 30 20:58:49.313778 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.313779 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" event={"ID":"9e463673-a152-4063-adba-af75c5e1b51a","Type":"ContainerStarted","Data":"2358ade52f2b03d29af09197e486c57d595f3169de2afbf98a6a4fef1b81db69"} Jul 30 20:58:49.315345 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.315322 2567 generic.go:356] "Generic (PLEG): container finished" podID="ddb0ab00-236d-45e1-896a-18270539a50b" containerID="b916c34cfbee5f011c6e8ca136b374e2b729b82448b2d2f26a95aba9bb90e1ae" exitCode=0 Jul 30 20:58:49.315476 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.315388 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnl62" event={"ID":"ddb0ab00-236d-45e1-896a-18270539a50b","Type":"ContainerDied","Data":"b916c34cfbee5f011c6e8ca136b374e2b729b82448b2d2f26a95aba9bb90e1ae"} Jul 30 20:58:49.316530 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.316513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"23a94640761bd1e98e6663d658d65c370e7ba19974ff4451eb5fc2bf90c28844"} Jul 30 20:58:49.334001 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.333953 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-nqg66" podStartSLOduration=1.421711782 podStartE2EDuration="3.333934124s" totalImagesPullingTime="1.912222342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:49.332362447 +0000 UTC m=+100.031498339" watchObservedRunningTime="2026-07-30 20:58:49.333934124 +0000 UTC m=+100.033070016" Jul 30 20:58:49.392428 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.392406 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77cz9\"" Jul 30 20:58:49.401138 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.401114 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:49.543476 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:49.543422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cbq8d"] Jul 30 20:58:49.545871 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:49.545846 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a05602_5dbd_4333_a5e1_bc095bb40f1e.slice/crio-0f472301d34df6e77a7fd0df415cddeafc450a2064c15085ab0bea10be4756f9 WatchSource:0}: Error finding container 0f472301d34df6e77a7fd0df415cddeafc450a2064c15085ab0bea10be4756f9: Status 404 returned error can't find the container with id 0f472301d34df6e77a7fd0df415cddeafc450a2064c15085ab0bea10be4756f9 Jul 30 20:58:50.324657 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.324119 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnl62" event={"ID":"ddb0ab00-236d-45e1-896a-18270539a50b","Type":"ContainerStarted","Data":"32a8f73c8aca632c6e13689534376cfe84eda3b706fd3790356a17cbeeb56ee7"} Jul 30 20:58:50.324657 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.324210 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnl62" event={"ID":"ddb0ab00-236d-45e1-896a-18270539a50b","Type":"ContainerStarted","Data":"9c0eebbe715a263ac02270ee887f1e9f733bfc111948a4611da34ce3419d0229"} Jul 30 20:58:50.325657 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.325617 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbq8d" event={"ID":"20a05602-5dbd-4333-a5e1-bc095bb40f1e","Type":"ContainerStarted","Data":"0f472301d34df6e77a7fd0df415cddeafc450a2064c15085ab0bea10be4756f9"} Jul 30 20:58:50.345797 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.345739 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lnl62" podStartSLOduration=2.874635436 podStartE2EDuration="4.345720503s" totalImagesPullingTime="1.396813053s" totalInitContainerRuntime="74.272014ms" isStatefulPod=true podCreationTimestamp="2026-07-30 20:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:50.344626015 +0000 UTC m=+101.043761906" watchObservedRunningTime="2026-07-30 20:58:50.345720503 +0000 UTC m=+101.044856394" Jul 30 20:58:50.752877 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.752848 2567 scope.go:117] "RemoveContainer" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" Jul 30 20:58:50.753054 ip-10-0-129-162 kubenswrapper[2567]: E0730 20:58:50.753029 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-cpjjc_openshift-console-operator(6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" podUID="6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31" Jul 30 20:58:50.865894 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.865864 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66cfdf6f9-ckdtm"] Jul 30 20:58:50.869095 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.869074 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871786 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871812 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871827 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871856 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wpgnq\"" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871789 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 30 20:58:50.872032 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.871862 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7e5ve32c1gncq\"" Jul 30 20:58:50.882309 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.882283 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66cfdf6f9-ckdtm"] Jul 30 20:58:50.942448 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942416 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-metrics-server-audit-profiles\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942611 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942464 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-client-certs\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942611 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942532 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2851a591-83a0-47de-911e-01c6e52fe25a-audit-log\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942611 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-tls\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942780 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942667 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7qgt\" (UniqueName: \"kubernetes.io/projected/2851a591-83a0-47de-911e-01c6e52fe25a-kube-api-access-p7qgt\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942780 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942713 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:50.942861 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:50.942782 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-client-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043531 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043448 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7qgt\" (UniqueName: \"kubernetes.io/projected/2851a591-83a0-47de-911e-01c6e52fe25a-kube-api-access-p7qgt\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043531 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043758 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-client-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043758 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043618 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-metrics-server-audit-profiles\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043758 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043653 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-client-certs\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043758 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2851a591-83a0-47de-911e-01c6e52fe25a-audit-log\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.043958 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.043853 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-tls\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.044124 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.044101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2851a591-83a0-47de-911e-01c6e52fe25a-audit-log\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.044485 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.044424 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.044821 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.044800 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2851a591-83a0-47de-911e-01c6e52fe25a-metrics-server-audit-profiles\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.046469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.046449 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-tls\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.046813 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.046793 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-secret-metrics-server-client-certs\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.046870 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.046794 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2851a591-83a0-47de-911e-01c6e52fe25a-client-ca-bundle\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.051518 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.051500 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7qgt\" (UniqueName: \"kubernetes.io/projected/2851a591-83a0-47de-911e-01c6e52fe25a-kube-api-access-p7qgt\") pod \"metrics-server-66cfdf6f9-ckdtm\" (UID: \"2851a591-83a0-47de-911e-01c6e52fe25a\") " pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.179424 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.179383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:58:51.853504 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:51.852078 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66cfdf6f9-ckdtm"] Jul 30 20:58:51.858878 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:58:51.858834 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2851a591_83a0_47de_911e_01c6e52fe25a.slice/crio-3b2c04b965d0ebf565be1e6016b0a95afe13b2cf41ea8485c1fdc06fcedd6a44 WatchSource:0}: Error finding container 3b2c04b965d0ebf565be1e6016b0a95afe13b2cf41ea8485c1fdc06fcedd6a44: Status 404 returned error can't find the container with id 3b2c04b965d0ebf565be1e6016b0a95afe13b2cf41ea8485c1fdc06fcedd6a44 Jul 30 20:58:52.335255 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.335216 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"6845f53e9b45e32d54157f8a3699782e1fb9a42ed7c1a9405dce16a68df2b221"} Jul 30 20:58:52.335413 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.335288 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"92c10714ed0c8d15bb0045f86245ad618ff24a87ac476a26bb2ee04e680cf743"} Jul 30 20:58:52.335413 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.335321 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"05606a4da4cdb7436c82993adf06f0e2058236c805adba10de74165d6043a2ed"} Jul 30 20:58:52.336275 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.336250 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" event={"ID":"2851a591-83a0-47de-911e-01c6e52fe25a","Type":"ContainerStarted","Data":"3b2c04b965d0ebf565be1e6016b0a95afe13b2cf41ea8485c1fdc06fcedd6a44"} Jul 30 20:58:52.337862 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.337839 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbq8d" event={"ID":"20a05602-5dbd-4333-a5e1-bc095bb40f1e","Type":"ContainerStarted","Data":"be9b16bfab6772c4773b6a89ab4f94134b10f429a2d75cbe7f673f75e3ad41ab"} Jul 30 20:58:52.337951 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.337878 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbq8d" event={"ID":"20a05602-5dbd-4333-a5e1-bc095bb40f1e","Type":"ContainerStarted","Data":"e5229312ffa2ef802bc5041625271d815f45329fc26ee098a3dd81261f06e8c0"} Jul 30 20:58:52.338116 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.338099 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cbq8d" Jul 30 20:58:52.354874 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:52.353359 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cbq8d" podStartSLOduration=65.187221191 podStartE2EDuration="1m7.353269767s" totalImagesPullingTime="2.166048576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:52.353023889 +0000 UTC m=+103.052159790" watchObservedRunningTime="2026-07-30 20:58:52.353269767 +0000 UTC m=+103.052405658" Jul 30 20:58:54.348808 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.348772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" event={"ID":"2851a591-83a0-47de-911e-01c6e52fe25a","Type":"ContainerStarted","Data":"beab27bc72861bf0d5592126255d6b06c53a6e4ff8e4e1c26a6bd13c0713b93c"} Jul 30 20:58:54.351841 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.351805 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"5aea364b34590743af85f805f37b8a583fe8fde50feeb56e075f0c64b706cf71"} Jul 30 20:58:54.352020 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.351858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"6e16bfa64e7b6af6deff26f5b660e71e580204c90c846099e9ce221de8caaad4"} Jul 30 20:58:54.352020 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.351893 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" event={"ID":"6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b","Type":"ContainerStarted","Data":"c138276198d9e1701c7c58cb077845ff362ab61c2a52b91d97607c4eb30ebd9c"} Jul 30 20:58:54.352020 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.351959 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:58:54.365905 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.365855 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" podStartSLOduration=2.722300121 podStartE2EDuration="4.365837476s" totalImagesPullingTime="1.643537355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:54.36476788 +0000 UTC m=+105.063903767" watchObservedRunningTime="2026-07-30 20:58:54.365837476 +0000 UTC m=+105.064973366" Jul 30 20:58:54.386252 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:58:54.386201 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" podStartSLOduration=2.04227891 podStartE2EDuration="6.386179842s" totalImagesPullingTime="4.343900932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:54.384932731 +0000 UTC m=+105.084068620" watchObservedRunningTime="2026-07-30 20:58:54.386179842 +0000 UTC m=+105.085315733" Jul 30 20:59:00.362355 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:00.362325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-598977cc7d-hfgvx" Jul 30 20:59:02.345502 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:02.345467 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cbq8d" Jul 30 20:59:04.752343 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:04.752315 2567 scope.go:117] "RemoveContainer" containerID="5fc0e38e3f7aa272728c9042ba0c736a95172877b345555eb72105f8b60fc6dd" Jul 30 20:59:05.396276 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.396248 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 20:59:05.396463 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.396339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" event={"ID":"6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31","Type":"ContainerStarted","Data":"0741ab74527a898a545a28d2241da35a41321cde61898ac527ce8db3fa975315"} Jul 30 20:59:05.396635 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.396613 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:59:05.510558 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.510533 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-cpjjc" Jul 30 20:59:05.714515 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.714445 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-sz96w"] Jul 30 20:59:05.716751 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.716731 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:05.718733 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.718710 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 20:59:05.718838 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.718811 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-p7hwj\"" Jul 30 20:59:05.718947 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.718932 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 20:59:05.730046 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.730027 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-sz96w"] Jul 30 20:59:05.873410 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.873380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2khl5\" (UniqueName: \"kubernetes.io/projected/3f2d4ab7-8141-415a-a91e-5554bc15aace-kube-api-access-2khl5\") pod \"downloads-77c974ffcf-sz96w\" (UID: \"3f2d4ab7-8141-415a-a91e-5554bc15aace\") " pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:05.974478 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.974403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2khl5\" (UniqueName: \"kubernetes.io/projected/3f2d4ab7-8141-415a-a91e-5554bc15aace-kube-api-access-2khl5\") pod \"downloads-77c974ffcf-sz96w\" (UID: \"3f2d4ab7-8141-415a-a91e-5554bc15aace\") " pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:05.982670 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:05.982646 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khl5\" (UniqueName: \"kubernetes.io/projected/3f2d4ab7-8141-415a-a91e-5554bc15aace-kube-api-access-2khl5\") pod \"downloads-77c974ffcf-sz96w\" (UID: \"3f2d4ab7-8141-415a-a91e-5554bc15aace\") " pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:06.025347 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:06.025323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:06.148930 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:06.148835 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-sz96w"] Jul 30 20:59:06.151378 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:59:06.151348 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2d4ab7_8141_415a_a91e_5554bc15aace.slice/crio-dd9ae946b085421334867880e254937d52b48146b86e9c8bd8199d9aa203bc96 WatchSource:0}: Error finding container dd9ae946b085421334867880e254937d52b48146b86e9c8bd8199d9aa203bc96: Status 404 returned error can't find the container with id dd9ae946b085421334867880e254937d52b48146b86e9c8bd8199d9aa203bc96 Jul 30 20:59:06.400802 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:06.400771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-sz96w" event={"ID":"3f2d4ab7-8141-415a-a91e-5554bc15aace","Type":"ContainerStarted","Data":"dd9ae946b085421334867880e254937d52b48146b86e9c8bd8199d9aa203bc96"} Jul 30 20:59:11.180521 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:11.180483 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:59:11.181058 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:11.180594 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:59:16.743295 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.743253 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 20:59:16.746309 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.746284 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.748487 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.748459 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 20:59:16.748630 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.748459 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 20:59:16.749048 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.749026 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 20:59:16.749157 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.749027 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 20:59:16.749157 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.749032 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xwhg9\"" Jul 30 20:59:16.749157 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.749032 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 20:59:16.757991 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.757956 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 20:59:16.772846 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.772821 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdqvc\" (UniqueName: \"kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.772963 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.772865 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.772963 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.772930 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.773072 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.773026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.773127 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.773091 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.773171 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.773154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874333 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874298 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdqvc\" (UniqueName: \"kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874510 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874342 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874510 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874391 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874510 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874510 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.874772 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.874530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.875198 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.875159 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.875321 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.875201 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.875321 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.875219 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.877344 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.877324 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.877472 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.877435 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:16.882423 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:16.882401 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdqvc\" (UniqueName: \"kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc\") pod \"console-6b87468bb-tnt25\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:17.057194 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:17.057156 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:19.497808 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:19.497775 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:59:19.500400 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:19.500373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8ad8626-c218-4c3f-918d-4295881c08e3-metrics-certs\") pod \"network-metrics-daemon-qct22\" (UID: \"f8ad8626-c218-4c3f-918d-4295881c08e3\") " pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:59:19.763352 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:19.763321 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5pq\"" Jul 30 20:59:19.772082 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:19.772061 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qct22" Jul 30 20:59:22.097304 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.097279 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 20:59:22.113439 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.113413 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qct22"] Jul 30 20:59:22.120918 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:59:22.120886 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ad8626_c218_4c3f_918d_4295881c08e3.slice/crio-8f61f9d935f9a3cd4fda15a582e5d3b1ed0be0bfc6bd34c5393edf2bde634705 WatchSource:0}: Error finding container 8f61f9d935f9a3cd4fda15a582e5d3b1ed0be0bfc6bd34c5393edf2bde634705: Status 404 returned error can't find the container with id 8f61f9d935f9a3cd4fda15a582e5d3b1ed0be0bfc6bd34c5393edf2bde634705 Jul 30 20:59:22.468103 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.468050 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qct22" event={"ID":"f8ad8626-c218-4c3f-918d-4295881c08e3","Type":"ContainerStarted","Data":"8f61f9d935f9a3cd4fda15a582e5d3b1ed0be0bfc6bd34c5393edf2bde634705"} Jul 30 20:59:22.469625 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.469569 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87468bb-tnt25" event={"ID":"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed","Type":"ContainerStarted","Data":"a472c0ea5d60a01f38c4bb4f5f33a016b1b6a1e7e24e6f6ad37d5f526711767e"} Jul 30 20:59:22.471272 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.471246 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-sz96w" event={"ID":"3f2d4ab7-8141-415a-a91e-5554bc15aace","Type":"ContainerStarted","Data":"5800e2e2d2073dab10894ead37c4493698a505c92aa6612c9a2d22fe61e6f49e"} Jul 30 20:59:22.471480 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.471460 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:22.488277 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.488250 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-sz96w" Jul 30 20:59:22.488805 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:22.488727 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-sz96w" podStartSLOduration=1.569818631 podStartE2EDuration="17.488708389s" totalImagesPullingTime="15.918889758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:22.488532281 +0000 UTC m=+133.187668172" watchObservedRunningTime="2026-07-30 20:59:22.488708389 +0000 UTC m=+133.187844276" Jul 30 20:59:24.484511 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:24.484425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qct22" event={"ID":"f8ad8626-c218-4c3f-918d-4295881c08e3","Type":"ContainerStarted","Data":"cca35c6cee1d5714c04d90eaa697c02dbb91a5a1b1086a26fca0fc23e60ffec8"} Jul 30 20:59:24.484511 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:24.484502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qct22" event={"ID":"f8ad8626-c218-4c3f-918d-4295881c08e3","Type":"ContainerStarted","Data":"69f3f2db9af519a671a176bbcbab6fe113447a14ca93dfed2e0e89bae6711474"} Jul 30 20:59:24.500660 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:24.500606 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qct22" podStartSLOduration=132.687016157 podStartE2EDuration="2m14.50056358s" totalImagesPullingTime="1.813547423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:24.499365774 +0000 UTC m=+135.198501664" watchObservedRunningTime="2026-07-30 20:59:24.50056358 +0000 UTC m=+135.199699471" Jul 30 20:59:25.824633 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.824597 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 20:59:25.850734 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.850704 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 20:59:25.850896 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.850838 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.858326 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.857933 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 30 20:59:25.954469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954422 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954469 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954469 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954690 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954528 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw6s5\" (UniqueName: \"kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954690 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954690 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954690 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954664 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:25.954869 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:25.954706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.055928 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.055880 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.055944 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.055981 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw6s5\" (UniqueName: \"kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056101 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056352 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056801 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056743 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.056959 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056937 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.057044 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.056996 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.057452 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.057425 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.058976 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.058951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.059618 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.059572 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.064464 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.064445 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw6s5\" (UniqueName: \"kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5\") pod \"console-96844784d-mk9tm\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.162869 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.162800 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:26.380255 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.380225 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 20:59:26.384229 ip-10-0-129-162 kubenswrapper[2567]: W0730 20:59:26.384202 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbe6b37_4d73_4420_a558_fdb102b29819.slice/crio-2640094e7455a109e054f69b06d43eb0b311f8c56c0338584dffd646e01da5fa WatchSource:0}: Error finding container 2640094e7455a109e054f69b06d43eb0b311f8c56c0338584dffd646e01da5fa: Status 404 returned error can't find the container with id 2640094e7455a109e054f69b06d43eb0b311f8c56c0338584dffd646e01da5fa Jul 30 20:59:26.496131 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.496094 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96844784d-mk9tm" event={"ID":"4fbe6b37-4d73-4420-a558-fdb102b29819","Type":"ContainerStarted","Data":"2640094e7455a109e054f69b06d43eb0b311f8c56c0338584dffd646e01da5fa"} Jul 30 20:59:26.497969 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.497939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87468bb-tnt25" event={"ID":"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed","Type":"ContainerStarted","Data":"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2"} Jul 30 20:59:26.515206 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.514334 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b87468bb-tnt25" podStartSLOduration=6.385046795 podStartE2EDuration="10.514315032s" totalImagesPullingTime="4.129268237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:26.513565119 +0000 UTC m=+137.212701009" watchObservedRunningTime="2026-07-30 20:59:26.514315032 +0000 UTC m=+137.213450923" Jul 30 20:59:26.647027 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:26.646993 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmdcg_3369fd3e-90aa-4e83-a9c5-b10b171b7d36/serve-healthcheck-canary/0.log" Jul 30 20:59:27.057718 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.057682 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:27.058157 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.057771 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:27.063322 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.063299 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:27.504402 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.504366 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96844784d-mk9tm" event={"ID":"4fbe6b37-4d73-4420-a558-fdb102b29819","Type":"ContainerStarted","Data":"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829"} Jul 30 20:59:27.509232 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.509174 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 20:59:27.524767 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:27.524703 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-96844784d-mk9tm" podStartSLOduration=2.524686152 podStartE2EDuration="2.524686152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:27.523211591 +0000 UTC m=+138.222347518" watchObservedRunningTime="2026-07-30 20:59:27.524686152 +0000 UTC m=+138.223822107" Jul 30 20:59:31.185522 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:31.185491 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:59:31.189414 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:31.189390 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66cfdf6f9-ckdtm" Jul 30 20:59:36.163368 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:36.163336 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:36.163887 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:36.163396 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:36.168119 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:36.168098 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:36.548257 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:36.548228 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-96844784d-mk9tm" Jul 30 20:59:36.587731 ip-10-0-129-162 kubenswrapper[2567]: I0730 20:59:36.587704 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 21:00:01.607183 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.607121 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b87468bb-tnt25" podUID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" containerName="console" containerID="cri-o://69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2" gracePeriod=15 Jul 30 21:00:01.868571 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.868502 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 21:00:01.943482 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943447 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.943704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943492 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.943704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943526 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdqvc\" (UniqueName: \"kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.943704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943549 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.943704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943606 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.943704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943638 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert\") pod \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\" (UID: \"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed\") " Jul 30 21:00:01.944016 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.943986 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca" (OuterVolumeSpecName: "service-ca") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:01.944149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.944057 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:01.944149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.944120 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config" (OuterVolumeSpecName: "console-config") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:01.945946 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.945925 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:00:01.945946 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.945930 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc" (OuterVolumeSpecName: "kube-api-access-kdqvc") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "kube-api-access-kdqvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:00:01.946063 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:01.945947 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" (UID: "18f58e87-a8c9-426c-b61c-2ea6fe9bdbed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:00:02.044401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044358 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-oauth-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.044401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044393 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.044401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044403 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-oauth-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.044401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044412 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-service-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.044696 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044421 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdqvc\" (UniqueName: \"kubernetes.io/projected/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-kube-api-access-kdqvc\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.044696 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.044430 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed-console-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:02.645829 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.645796 2567 generic.go:356] "Generic (PLEG): container finished" podID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" containerID="69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2" exitCode=0 Jul 30 21:00:02.646234 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.645840 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87468bb-tnt25" Jul 30 21:00:02.646234 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.645861 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87468bb-tnt25" event={"ID":"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed","Type":"ContainerDied","Data":"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2"} Jul 30 21:00:02.646234 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.645901 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87468bb-tnt25" event={"ID":"18f58e87-a8c9-426c-b61c-2ea6fe9bdbed","Type":"ContainerDied","Data":"a472c0ea5d60a01f38c4bb4f5f33a016b1b6a1e7e24e6f6ad37d5f526711767e"} Jul 30 21:00:02.646234 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.645920 2567 scope.go:117] "RemoveContainer" containerID="69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2" Jul 30 21:00:02.654528 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.654500 2567 scope.go:117] "RemoveContainer" containerID="69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2" Jul 30 21:00:02.654789 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:00:02.654770 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2\": container with ID starting with 69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2 not found: ID does not exist" containerID="69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2" Jul 30 21:00:02.654842 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.654798 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2"} err="failed to get container status \"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2\": rpc error: code = NotFound desc = could not find container \"69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2\": container with ID starting with 69776c4d67e6a08bf1c0e139a4e130936d5ae97328223396ac7c95a75b9795a2 not found: ID does not exist" Jul 30 21:00:02.665967 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.665943 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 21:00:02.676448 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:02.673649 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b87468bb-tnt25"] Jul 30 21:00:03.458261 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.458226 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:00:03.458742 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.458719 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" containerName="console" Jul 30 21:00:03.458865 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.458746 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" containerName="console" Jul 30 21:00:03.458865 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.458830 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" containerName="console" Jul 30 21:00:03.491375 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.491351 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:00:03.491505 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.491439 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554453 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554421 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554453 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554454 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554681 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554681 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554681 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554621 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554681 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ss7\" (UniqueName: \"kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.554825 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.554690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655106 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ss7\" (UniqueName: \"kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655159 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655174 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655249 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.655531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.656020 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.655987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.656084 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.656031 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.656084 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.656066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.656214 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.656196 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.657957 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.657928 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.658047 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.657941 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.662600 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.662558 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ss7\" (UniqueName: \"kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7\") pod \"console-7c57b46c86-kztvb\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.758137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.758110 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f58e87-a8c9-426c-b61c-2ea6fe9bdbed" path="/var/lib/kubelet/pods/18f58e87-a8c9-426c-b61c-2ea6fe9bdbed/volumes" Jul 30 21:00:03.800209 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.800180 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:03.961051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:03.960996 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:00:03.963901 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:00:03.963871 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99513a5b_f2bc_4c59_9e7d_0da56f1bd2fa.slice/crio-07dbeb21b572fa83feaca1173d14a8e36ac6c8b158f5b88e75674fee9022d152 WatchSource:0}: Error finding container 07dbeb21b572fa83feaca1173d14a8e36ac6c8b158f5b88e75674fee9022d152: Status 404 returned error can't find the container with id 07dbeb21b572fa83feaca1173d14a8e36ac6c8b158f5b88e75674fee9022d152 Jul 30 21:00:04.658004 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:04.657964 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57b46c86-kztvb" event={"ID":"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa","Type":"ContainerStarted","Data":"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4"} Jul 30 21:00:04.658393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:04.658023 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57b46c86-kztvb" event={"ID":"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa","Type":"ContainerStarted","Data":"07dbeb21b572fa83feaca1173d14a8e36ac6c8b158f5b88e75674fee9022d152"} Jul 30 21:00:04.674308 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:04.674261 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c57b46c86-kztvb" podStartSLOduration=1.674246025 podStartE2EDuration="1.674246025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:00:04.673255087 +0000 UTC m=+175.372390977" watchObservedRunningTime="2026-07-30 21:00:04.674246025 +0000 UTC m=+175.373381914" Jul 30 21:00:13.800779 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:13.800743 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:13.801238 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:13.800796 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:13.805618 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:13.805598 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:14.707277 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:14.707250 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:00:14.750654 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:14.750617 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 21:00:39.770245 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:39.770198 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-96844784d-mk9tm" podUID="4fbe6b37-4d73-4420-a558-fdb102b29819" containerName="console" containerID="cri-o://634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829" gracePeriod=15 Jul 30 21:00:39.799327 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:00:39.799301 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbe6b37_4d73_4420_a558_fdb102b29819.slice/crio-634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829.scope\": RecentStats: unable to find data in memory cache]" Jul 30 21:00:40.002810 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.002786 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96844784d-mk9tm" Jul 30 21:00:40.112005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.111922 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.111964 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.111992 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112271 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112025 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112271 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112071 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw6s5\" (UniqueName: \"kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112271 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112100 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112271 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112150 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle\") pod \"4fbe6b37-4d73-4420-a558-fdb102b29819\" (UID: \"4fbe6b37-4d73-4420-a558-fdb102b29819\") " Jul 30 21:00:40.112467 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112373 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:40.112630 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112501 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca" (OuterVolumeSpecName: "service-ca") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:40.112716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112626 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config" (OuterVolumeSpecName: "console-config") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:40.112716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.112671 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:00:40.114251 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.114230 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5" (OuterVolumeSpecName: "kube-api-access-cw6s5") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "kube-api-access-cw6s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:00:40.114555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.114522 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:00:40.114555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.114533 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4fbe6b37-4d73-4420-a558-fdb102b29819" (UID: "4fbe6b37-4d73-4420-a558-fdb102b29819"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:00:40.213356 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213325 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-oauth-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213356 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213349 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213356 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213358 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fbe6b37-4d73-4420-a558-fdb102b29819-console-oauth-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213367 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-service-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213376 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw6s5\" (UniqueName: \"kubernetes.io/projected/4fbe6b37-4d73-4420-a558-fdb102b29819-kube-api-access-cw6s5\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213384 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-console-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.213567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.213393 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fbe6b37-4d73-4420-a558-fdb102b29819-trusted-ca-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:00:40.799536 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.799500 2567 generic.go:356] "Generic (PLEG): container finished" podID="4fbe6b37-4d73-4420-a558-fdb102b29819" containerID="634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829" exitCode=0 Jul 30 21:00:40.800030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.799544 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96844784d-mk9tm" Jul 30 21:00:40.800030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.799622 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96844784d-mk9tm" event={"ID":"4fbe6b37-4d73-4420-a558-fdb102b29819","Type":"ContainerDied","Data":"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829"} Jul 30 21:00:40.800030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.799669 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96844784d-mk9tm" event={"ID":"4fbe6b37-4d73-4420-a558-fdb102b29819","Type":"ContainerDied","Data":"2640094e7455a109e054f69b06d43eb0b311f8c56c0338584dffd646e01da5fa"} Jul 30 21:00:40.800030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.799690 2567 scope.go:117] "RemoveContainer" containerID="634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829" Jul 30 21:00:40.808083 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.808067 2567 scope.go:117] "RemoveContainer" containerID="634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829" Jul 30 21:00:40.808322 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:00:40.808303 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829\": container with ID starting with 634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829 not found: ID does not exist" containerID="634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829" Jul 30 21:00:40.808386 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.808330 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829"} err="failed to get container status \"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829\": rpc error: code = NotFound desc = could not find container \"634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829\": container with ID starting with 634c0d8fcbe4d6773f205d4dda7d0718d1e0ee6fae5899b68f4785baea2cc829 not found: ID does not exist" Jul 30 21:00:40.818908 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.818886 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 21:00:40.822418 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:40.822399 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-96844784d-mk9tm"] Jul 30 21:00:41.757144 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:41.757110 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fbe6b37-4d73-4420-a558-fdb102b29819" path="/var/lib/kubelet/pods/4fbe6b37-4d73-4420-a558-fdb102b29819/volumes" Jul 30 21:00:56.842941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.842907 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh"] Jul 30 21:00:56.843300 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.843251 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fbe6b37-4d73-4420-a558-fdb102b29819" containerName="console" Jul 30 21:00:56.843300 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.843263 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fbe6b37-4d73-4420-a558-fdb102b29819" containerName="console" Jul 30 21:00:56.843374 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.843328 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fbe6b37-4d73-4420-a558-fdb102b29819" containerName="console" Jul 30 21:00:56.846571 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.846553 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:56.848810 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.848788 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 21:00:56.849237 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.849222 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k4jhp\"" Jul 30 21:00:56.849307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.849223 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 21:00:56.854471 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.854447 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh"] Jul 30 21:00:56.937659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.937631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgslc\" (UniqueName: \"kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:56.937789 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.937672 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:56.937839 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:56.937795 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.038666 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.038632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgslc\" (UniqueName: \"kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.038821 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.038673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.038884 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.038860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.039015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.039000 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.039149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.039134 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.046103 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.046074 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgslc\" (UniqueName: \"kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.156513 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.156447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:00:57.277706 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.277591 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh"] Jul 30 21:00:57.279977 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:00:57.279945 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8328ab55_a5cf_415b_ad22_dd71264a605d.slice/crio-bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4 WatchSource:0}: Error finding container bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4: Status 404 returned error can't find the container with id bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4 Jul 30 21:00:57.861802 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:00:57.861772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" event={"ID":"8328ab55-a5cf-415b-ad22-dd71264a605d","Type":"ContainerStarted","Data":"bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4"} Jul 30 21:01:04.891409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:04.891375 2567 generic.go:356] "Generic (PLEG): container finished" podID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerID="f9364959604c2754cc45b025389075a1d3bf9aafc32bf6a997484b8651561dec" exitCode=0 Jul 30 21:01:04.891872 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:04.891417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" event={"ID":"8328ab55-a5cf-415b-ad22-dd71264a605d","Type":"ContainerDied","Data":"f9364959604c2754cc45b025389075a1d3bf9aafc32bf6a997484b8651561dec"} Jul 30 21:01:07.904832 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:07.904801 2567 generic.go:356] "Generic (PLEG): container finished" podID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerID="0ab8a04dcb1924425c0baacfe3e40ae6780495e7bdc1331e47031eb56926e926" exitCode=0 Jul 30 21:01:07.905209 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:07.904848 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" event={"ID":"8328ab55-a5cf-415b-ad22-dd71264a605d","Type":"ContainerDied","Data":"0ab8a04dcb1924425c0baacfe3e40ae6780495e7bdc1331e47031eb56926e926"} Jul 30 21:01:19.956601 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:19.956500 2567 generic.go:356] "Generic (PLEG): container finished" podID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerID="7adc7278c25f1ae1ff07894186cc788e9c672824db5c736c7082bc9916568439" exitCode=0 Jul 30 21:01:19.956967 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:19.956608 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" event={"ID":"8328ab55-a5cf-415b-ad22-dd71264a605d","Type":"ContainerDied","Data":"7adc7278c25f1ae1ff07894186cc788e9c672824db5c736c7082bc9916568439"} Jul 30 21:01:21.114662 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.114633 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:01:21.135621 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.135593 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util\") pod \"8328ab55-a5cf-415b-ad22-dd71264a605d\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " Jul 30 21:01:21.135760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.135641 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle\") pod \"8328ab55-a5cf-415b-ad22-dd71264a605d\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " Jul 30 21:01:21.135760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.135674 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgslc\" (UniqueName: \"kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc\") pod \"8328ab55-a5cf-415b-ad22-dd71264a605d\" (UID: \"8328ab55-a5cf-415b-ad22-dd71264a605d\") " Jul 30 21:01:21.136388 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.136358 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle" (OuterVolumeSpecName: "bundle") pod "8328ab55-a5cf-415b-ad22-dd71264a605d" (UID: "8328ab55-a5cf-415b-ad22-dd71264a605d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:01:21.137862 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.137834 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc" (OuterVolumeSpecName: "kube-api-access-wgslc") pod "8328ab55-a5cf-415b-ad22-dd71264a605d" (UID: "8328ab55-a5cf-415b-ad22-dd71264a605d"). InnerVolumeSpecName "kube-api-access-wgslc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:01:21.140905 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.140884 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util" (OuterVolumeSpecName: "util") pod "8328ab55-a5cf-415b-ad22-dd71264a605d" (UID: "8328ab55-a5cf-415b-ad22-dd71264a605d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:01:21.237124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.237072 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:21.237124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.237093 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8328ab55-a5cf-415b-ad22-dd71264a605d-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:21.237124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.237102 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgslc\" (UniqueName: \"kubernetes.io/projected/8328ab55-a5cf-415b-ad22-dd71264a605d-kube-api-access-wgslc\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:21.967809 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.967781 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" Jul 30 21:01:21.967958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.967820 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sggsh" event={"ID":"8328ab55-a5cf-415b-ad22-dd71264a605d","Type":"ContainerDied","Data":"bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4"} Jul 30 21:01:21.967958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:21.967863 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb288a1e1b72ab4776a156c797f61b1542ab467a38c0919b7fe70db3f49344e4" Jul 30 21:01:29.897678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897643 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb"] Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897927 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="pull" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897937 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="pull" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897946 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="extract" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897952 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="extract" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897960 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="util" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.897966 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="util" Jul 30 21:01:29.898021 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.898011 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8328ab55-a5cf-415b-ad22-dd71264a605d" containerName="extract" Jul 30 21:01:29.899866 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.899851 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:29.901880 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.901857 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 30 21:01:29.902004 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.901861 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rhlxm\"" Jul 30 21:01:29.902004 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.901863 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 30 21:01:29.911210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.911188 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb"] Jul 30 21:01:29.999606 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.999568 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9pv\" (UniqueName: \"kubernetes.io/projected/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-kube-api-access-4c9pv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:29.999735 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:29.999618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.100511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.100474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9pv\" (UniqueName: \"kubernetes.io/projected/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-kube-api-access-4c9pv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.100511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.100516 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.100874 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.100856 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.108927 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.108902 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9pv\" (UniqueName: \"kubernetes.io/projected/c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e-kube-api-access-4c9pv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5whcb\" (UID: \"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.208930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.208868 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" Jul 30 21:01:30.342556 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:30.342522 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb"] Jul 30 21:01:30.345119 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:01:30.345094 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cd3ecd_f0d3_4db9_b785_58b4a38fea0e.slice/crio-6edd9883f2cf1b8d34303602c9998ba485d260ebef8d276f7dac9024bf98654b WatchSource:0}: Error finding container 6edd9883f2cf1b8d34303602c9998ba485d260ebef8d276f7dac9024bf98654b: Status 404 returned error can't find the container with id 6edd9883f2cf1b8d34303602c9998ba485d260ebef8d276f7dac9024bf98654b Jul 30 21:01:31.004351 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:31.004311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" event={"ID":"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e","Type":"ContainerStarted","Data":"6edd9883f2cf1b8d34303602c9998ba485d260ebef8d276f7dac9024bf98654b"} Jul 30 21:01:44.069086 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:44.069050 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" event={"ID":"c4cd3ecd-f0d3-4db9-b785-58b4a38fea0e","Type":"ContainerStarted","Data":"7b8866b8c0b3d9a07daa654aeab9c3752446a651618d9f48a5cb9d843f4928e6"} Jul 30 21:01:44.096198 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:44.096142 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5whcb" podStartSLOduration=1.779564944 podStartE2EDuration="15.096126776s" totalImagesPullingTime="13.316561832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:01:44.093904978 +0000 UTC m=+274.793040879" watchObservedRunningTime="2026-07-30 21:01:44.096126776 +0000 UTC m=+274.795262665" Jul 30 21:01:45.600677 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.600644 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q"] Jul 30 21:01:45.602988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.602972 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.605678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.605658 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k4jhp\"" Jul 30 21:01:45.606174 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.606162 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 21:01:45.606454 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.606440 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 21:01:45.613769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.613752 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q"] Jul 30 21:01:45.726670 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.726643 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2vgl\" (UniqueName: \"kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.726813 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.726699 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.726813 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.726760 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.827423 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.827397 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.827602 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.827435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.827602 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.827475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2vgl\" (UniqueName: \"kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.827821 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.827792 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.827882 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.827866 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.835558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.835532 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2vgl\" (UniqueName: \"kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:45.912072 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:45.912018 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:46.057299 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:46.057267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q"] Jul 30 21:01:46.060614 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:01:46.060588 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc90f836_b48e_45a5_b160_56393dbc4f85.slice/crio-9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da WatchSource:0}: Error finding container 9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da: Status 404 returned error can't find the container with id 9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da Jul 30 21:01:46.078463 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:46.078443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" event={"ID":"bc90f836-b48e-45a5-b160-56393dbc4f85","Type":"ContainerStarted","Data":"9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da"} Jul 30 21:01:47.084456 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.084426 2567 generic.go:356] "Generic (PLEG): container finished" podID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerID="eccefae9652d342be6107ecd2bfed08cf29abbf1386432dde6150098eec00859" exitCode=0 Jul 30 21:01:47.084846 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.084516 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" event={"ID":"bc90f836-b48e-45a5-b160-56393dbc4f85","Type":"ContainerDied","Data":"eccefae9652d342be6107ecd2bfed08cf29abbf1386432dde6150098eec00859"} Jul 30 21:01:47.928818 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.928781 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd"] Jul 30 21:01:47.931031 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.931008 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:47.933246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.933207 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 30 21:01:47.933246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.933227 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bxlhn\"" Jul 30 21:01:47.933365 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.933299 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 30 21:01:47.939294 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:47.939273 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd"] Jul 30 21:01:48.041521 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.041491 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644s5\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-kube-api-access-644s5\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.041672 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.041543 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.142088 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.142060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644s5\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-kube-api-access-644s5\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.142440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.142120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.149839 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.149817 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.150082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.150065 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-644s5\" (UniqueName: \"kubernetes.io/projected/d2f0380b-5c05-4ecb-af77-0fef5615ce63-kube-api-access-644s5\") pod \"cert-manager-cainjector-559cdd8c87-2mjpd\" (UID: \"d2f0380b-5c05-4ecb-af77-0fef5615ce63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.263327 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.263289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" Jul 30 21:01:48.385619 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:48.385595 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd"] Jul 30 21:01:48.387659 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:01:48.387633 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2f0380b_5c05_4ecb_af77_0fef5615ce63.slice/crio-8f04a42e5c556ce65ffebc8a1e7872070ad249965b54f591d1c75d959ad4aee9 WatchSource:0}: Error finding container 8f04a42e5c556ce65ffebc8a1e7872070ad249965b54f591d1c75d959ad4aee9: Status 404 returned error can't find the container with id 8f04a42e5c556ce65ffebc8a1e7872070ad249965b54f591d1c75d959ad4aee9 Jul 30 21:01:49.099279 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:49.099242 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" event={"ID":"d2f0380b-5c05-4ecb-af77-0fef5615ce63","Type":"ContainerStarted","Data":"8f04a42e5c556ce65ffebc8a1e7872070ad249965b54f591d1c75d959ad4aee9"} Jul 30 21:01:52.687116 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.687027 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q72dt"] Jul 30 21:01:52.689224 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.689196 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.691275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.691249 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kvgz4\"" Jul 30 21:01:52.697666 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.697645 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q72dt"] Jul 30 21:01:52.783491 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.783447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hkd4\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-kube-api-access-4hkd4\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.783685 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.783546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.884760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.884720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hkd4\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-kube-api-access-4hkd4\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.885028 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.884998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.893154 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.893131 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.893289 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.893178 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hkd4\" (UniqueName: \"kubernetes.io/projected/61b42b3b-98a3-4a80-8afd-25816fce6906-kube-api-access-4hkd4\") pod \"cert-manager-webhook-6486ffd997-q72dt\" (UID: \"61b42b3b-98a3-4a80-8afd-25816fce6906\") " pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:52.998481 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:52.998456 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:53.126634 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:53.126594 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" event={"ID":"d2f0380b-5c05-4ecb-af77-0fef5615ce63","Type":"ContainerStarted","Data":"6e1ae8f2e3f794f21f888e3848d62bbb5f98a6ba484023d866a3e94513c26a9c"} Jul 30 21:01:53.127413 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:53.127261 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-q72dt"] Jul 30 21:01:53.128669 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:53.128643 2567 generic.go:356] "Generic (PLEG): container finished" podID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerID="117c816a383e0efd0f245a6a354c49a3383614e965f13d29afbcd5499440e2ee" exitCode=0 Jul 30 21:01:53.128769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:53.128735 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" event={"ID":"bc90f836-b48e-45a5-b160-56393dbc4f85","Type":"ContainerDied","Data":"117c816a383e0efd0f245a6a354c49a3383614e965f13d29afbcd5499440e2ee"} Jul 30 21:01:53.130175 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:01:53.130156 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61b42b3b_98a3_4a80_8afd_25816fce6906.slice/crio-dd566ab2a7071e7c14a5aac8527fdf4d5b50551c1ad839e4f40e674369b585ff WatchSource:0}: Error finding container dd566ab2a7071e7c14a5aac8527fdf4d5b50551c1ad839e4f40e674369b585ff: Status 404 returned error can't find the container with id dd566ab2a7071e7c14a5aac8527fdf4d5b50551c1ad839e4f40e674369b585ff Jul 30 21:01:53.141497 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:53.141449 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2mjpd" podStartSLOduration=2.115120083 podStartE2EDuration="6.14143156s" totalImagesPullingTime="4.026311477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:01:53.140932594 +0000 UTC m=+283.840068484" watchObservedRunningTime="2026-07-30 21:01:53.14143156 +0000 UTC m=+283.840567452" Jul 30 21:01:54.134192 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.134156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" event={"ID":"61b42b3b-98a3-4a80-8afd-25816fce6906","Type":"ContainerStarted","Data":"5333071cdf3cd0dfd253846f8ea28f83f762cc754d5d10732570777936b8f073"} Jul 30 21:01:54.134728 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.134210 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:01:54.134728 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.134225 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" event={"ID":"61b42b3b-98a3-4a80-8afd-25816fce6906","Type":"ContainerStarted","Data":"dd566ab2a7071e7c14a5aac8527fdf4d5b50551c1ad839e4f40e674369b585ff"} Jul 30 21:01:54.136197 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.136165 2567 generic.go:356] "Generic (PLEG): container finished" podID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerID="d19086727e8b912a47fb204bdc29ac57091a018ae7d5562adab4289d3566b72a" exitCode=0 Jul 30 21:01:54.136323 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.136232 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" event={"ID":"bc90f836-b48e-45a5-b160-56393dbc4f85","Type":"ContainerDied","Data":"d19086727e8b912a47fb204bdc29ac57091a018ae7d5562adab4289d3566b72a"} Jul 30 21:01:54.149658 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:54.149617 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" podStartSLOduration=2.149602872 podStartE2EDuration="2.149602872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:01:54.147934943 +0000 UTC m=+284.847070833" watchObservedRunningTime="2026-07-30 21:01:54.149602872 +0000 UTC m=+284.848738758" Jul 30 21:01:55.266024 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.265994 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:55.407301 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.407222 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2vgl\" (UniqueName: \"kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl\") pod \"bc90f836-b48e-45a5-b160-56393dbc4f85\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " Jul 30 21:01:55.407301 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.407259 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util\") pod \"bc90f836-b48e-45a5-b160-56393dbc4f85\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " Jul 30 21:01:55.407473 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.407312 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle\") pod \"bc90f836-b48e-45a5-b160-56393dbc4f85\" (UID: \"bc90f836-b48e-45a5-b160-56393dbc4f85\") " Jul 30 21:01:55.407747 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.407721 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle" (OuterVolumeSpecName: "bundle") pod "bc90f836-b48e-45a5-b160-56393dbc4f85" (UID: "bc90f836-b48e-45a5-b160-56393dbc4f85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:01:55.409306 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.409282 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl" (OuterVolumeSpecName: "kube-api-access-r2vgl") pod "bc90f836-b48e-45a5-b160-56393dbc4f85" (UID: "bc90f836-b48e-45a5-b160-56393dbc4f85"). InnerVolumeSpecName "kube-api-access-r2vgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:01:55.411459 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.411434 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util" (OuterVolumeSpecName: "util") pod "bc90f836-b48e-45a5-b160-56393dbc4f85" (UID: "bc90f836-b48e-45a5-b160-56393dbc4f85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:01:55.508358 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.508322 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:55.508358 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.508351 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2vgl\" (UniqueName: \"kubernetes.io/projected/bc90f836-b48e-45a5-b160-56393dbc4f85-kube-api-access-r2vgl\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:55.508358 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:55.508365 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc90f836-b48e-45a5-b160-56393dbc4f85-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:01:56.147655 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:56.147551 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" event={"ID":"bc90f836-b48e-45a5-b160-56393dbc4f85","Type":"ContainerDied","Data":"9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da"} Jul 30 21:01:56.147655 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:56.147575 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fccd2q" Jul 30 21:01:56.147655 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:56.147621 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9562f3d48d2725ecaa445fb6be753117212b11170b118b4e881d427927de28da" Jul 30 21:01:59.724680 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.724645 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl"] Jul 30 21:01:59.725059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.724991 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="extract" Jul 30 21:01:59.725059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725006 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="extract" Jul 30 21:01:59.725059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725039 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="util" Jul 30 21:01:59.725059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725046 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="util" Jul 30 21:01:59.725178 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725068 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="pull" Jul 30 21:01:59.725178 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725076 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="pull" Jul 30 21:01:59.725178 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.725123 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc90f836-b48e-45a5-b160-56393dbc4f85" containerName="extract" Jul 30 21:01:59.727402 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.727383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.729448 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.729424 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 30 21:01:59.730375 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.730359 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 30 21:01:59.730456 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.730392 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kjbc6\"" Jul 30 21:01:59.735526 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.735505 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl"] Jul 30 21:01:59.843509 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.843477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pb9z\" (UniqueName: \"kubernetes.io/projected/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-kube-api-access-5pb9z\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.843752 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.843536 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.944947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.944908 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.945121 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.944986 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pb9z\" (UniqueName: \"kubernetes.io/projected/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-kube-api-access-5pb9z\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.945320 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.945300 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:01:59.952511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:01:59.952462 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pb9z\" (UniqueName: \"kubernetes.io/projected/c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b-kube-api-access-5pb9z\") pod \"openshift-lws-operator-fd8ccff4c-jz2wl\" (UID: \"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:02:00.037343 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:00.037303 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" Jul 30 21:02:00.143755 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:00.143727 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-q72dt" Jul 30 21:02:00.172173 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:00.172142 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl"] Jul 30 21:02:00.174304 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:02:00.174279 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9b8f1bc_ca86_44d6_a0f7_aea7d8c4ed6b.slice/crio-1986a1e02945ee940116faabcebc715bd21dd7363d1b2c06050df16c1b3cb2fb WatchSource:0}: Error finding container 1986a1e02945ee940116faabcebc715bd21dd7363d1b2c06050df16c1b3cb2fb: Status 404 returned error can't find the container with id 1986a1e02945ee940116faabcebc715bd21dd7363d1b2c06050df16c1b3cb2fb Jul 30 21:02:01.178730 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:01.178692 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" event={"ID":"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b","Type":"ContainerStarted","Data":"1986a1e02945ee940116faabcebc715bd21dd7363d1b2c06050df16c1b3cb2fb"} Jul 30 21:02:06.202635 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:06.202520 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" event={"ID":"c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b","Type":"ContainerStarted","Data":"421b7a6e3c2cbb794087b78c2a456fce869a60b81e920894b4416cfc953cd5b2"} Jul 30 21:02:06.218263 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:06.218200 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jz2wl" podStartSLOduration=1.500146309 podStartE2EDuration="7.218183278s" totalImagesPullingTime="5.718036969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:06.217308693 +0000 UTC m=+296.916444581" watchObservedRunningTime="2026-07-30 21:02:06.218183278 +0000 UTC m=+296.917319172" Jul 30 21:02:08.516399 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.516366 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl"] Jul 30 21:02:08.518999 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.518984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.521351 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.521329 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 21:02:08.521459 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.521330 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 21:02:08.521678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.521663 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k4jhp\"" Jul 30 21:02:08.527077 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.527057 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl"] Jul 30 21:02:08.615993 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.615965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjl58\" (UniqueName: \"kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.616125 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.616026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.616125 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.616090 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.716397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.716371 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjl58\" (UniqueName: \"kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.716497 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.716433 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.716497 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.716451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.716782 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.716767 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.716841 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.716801 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.724612 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.724570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjl58\" (UniqueName: \"kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.828226 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.828172 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:08.954445 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:08.954419 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl"] Jul 30 21:02:08.956524 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:02:08.956493 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b6f5ec_df1f_449b_89ad_13d09895913d.slice/crio-a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0 WatchSource:0}: Error finding container a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0: Status 404 returned error can't find the container with id a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0 Jul 30 21:02:09.217838 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.217760 2567 generic.go:356] "Generic (PLEG): container finished" podID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerID="2d4ced97204205995684d5e932db557198ea1078c718d9a6ad5959ef267a8f16" exitCode=0 Jul 30 21:02:09.217838 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.217822 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" event={"ID":"00b6f5ec-df1f-449b-89ad-13d09895913d","Type":"ContainerDied","Data":"2d4ced97204205995684d5e932db557198ea1078c718d9a6ad5959ef267a8f16"} Jul 30 21:02:09.218014 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.217861 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" event={"ID":"00b6f5ec-df1f-449b-89ad-13d09895913d","Type":"ContainerStarted","Data":"a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0"} Jul 30 21:02:09.679741 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.679711 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:02:09.680210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.679863 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:02:09.690135 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:09.690114 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 21:02:10.225079 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:10.225036 2567 generic.go:356] "Generic (PLEG): container finished" podID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerID="800d3a2de90a3920b946321e2f5bc8aecae46c69a488cecbdbef560864159bbf" exitCode=0 Jul 30 21:02:10.225282 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:10.225107 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" event={"ID":"00b6f5ec-df1f-449b-89ad-13d09895913d","Type":"ContainerDied","Data":"800d3a2de90a3920b946321e2f5bc8aecae46c69a488cecbdbef560864159bbf"} Jul 30 21:02:11.231726 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:11.231690 2567 generic.go:356] "Generic (PLEG): container finished" podID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerID="ad6af508f6b9d256e0c82fc3a85f418a922fd8de7ba62dd88111e1571538bd6d" exitCode=0 Jul 30 21:02:11.232110 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:11.231770 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" event={"ID":"00b6f5ec-df1f-449b-89ad-13d09895913d","Type":"ContainerDied","Data":"ad6af508f6b9d256e0c82fc3a85f418a922fd8de7ba62dd88111e1571538bd6d"} Jul 30 21:02:12.361372 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.361347 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:12.447373 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.447346 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle\") pod \"00b6f5ec-df1f-449b-89ad-13d09895913d\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " Jul 30 21:02:12.447373 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.447376 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util\") pod \"00b6f5ec-df1f-449b-89ad-13d09895913d\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " Jul 30 21:02:12.447638 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.447418 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjl58\" (UniqueName: \"kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58\") pod \"00b6f5ec-df1f-449b-89ad-13d09895913d\" (UID: \"00b6f5ec-df1f-449b-89ad-13d09895913d\") " Jul 30 21:02:12.448113 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.448082 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle" (OuterVolumeSpecName: "bundle") pod "00b6f5ec-df1f-449b-89ad-13d09895913d" (UID: "00b6f5ec-df1f-449b-89ad-13d09895913d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:02:12.449478 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.449447 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58" (OuterVolumeSpecName: "kube-api-access-hjl58") pod "00b6f5ec-df1f-449b-89ad-13d09895913d" (UID: "00b6f5ec-df1f-449b-89ad-13d09895913d"). InnerVolumeSpecName "kube-api-access-hjl58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:02:12.453022 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.452980 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util" (OuterVolumeSpecName: "util") pod "00b6f5ec-df1f-449b-89ad-13d09895913d" (UID: "00b6f5ec-df1f-449b-89ad-13d09895913d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:02:12.548067 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.548035 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:02:12.548067 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.548062 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00b6f5ec-df1f-449b-89ad-13d09895913d-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:02:12.548225 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:12.548076 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjl58\" (UniqueName: \"kubernetes.io/projected/00b6f5ec-df1f-449b-89ad-13d09895913d-kube-api-access-hjl58\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:02:13.242618 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:13.242564 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" event={"ID":"00b6f5ec-df1f-449b-89ad-13d09895913d","Type":"ContainerDied","Data":"a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0"} Jul 30 21:02:13.242618 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:13.242611 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcxl" Jul 30 21:02:13.242810 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:13.242628 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a073cac1939a1a0eb1939f7d12eb0de8ceae654903cb5892cc5d9555e76f3cd0" Jul 30 21:02:19.820121 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820088 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9"] Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820378 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="pull" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820388 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="pull" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820400 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="util" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820405 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="util" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820414 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="extract" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820419 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="extract" Jul 30 21:02:19.820484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.820475 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b6f5ec-df1f-449b-89ad-13d09895913d" containerName="extract" Jul 30 21:02:19.826257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.826236 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:19.829767 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.829745 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 30 21:02:19.829890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.829749 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tvk9w\"" Jul 30 21:02:19.829890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.829812 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 30 21:02:19.829890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.829838 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 30 21:02:19.838669 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:19.838649 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9"] Jul 30 21:02:20.008844 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.008807 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ad83382-33cb-4dc1-a244-c85fbd8e4264-manager-config\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.008844 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.008845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-metrics-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.009131 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.008884 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.009131 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.008992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdht9\" (UniqueName: \"kubernetes.io/projected/7ad83382-33cb-4dc1-a244-c85fbd8e4264-kube-api-access-kdht9\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.109932 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.109837 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdht9\" (UniqueName: \"kubernetes.io/projected/7ad83382-33cb-4dc1-a244-c85fbd8e4264-kube-api-access-kdht9\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.109932 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.109891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ad83382-33cb-4dc1-a244-c85fbd8e4264-manager-config\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.110133 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.110002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-metrics-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.110133 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.110063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.110616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.110568 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ad83382-33cb-4dc1-a244-c85fbd8e4264-manager-config\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.112511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.112492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-metrics-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.112668 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.112648 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ad83382-33cb-4dc1-a244-c85fbd8e4264-cert\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.129160 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.129134 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdht9\" (UniqueName: \"kubernetes.io/projected/7ad83382-33cb-4dc1-a244-c85fbd8e4264-kube-api-access-kdht9\") pod \"lws-controller-manager-76d7f64c45-rvlb9\" (UID: \"7ad83382-33cb-4dc1-a244-c85fbd8e4264\") " pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.135093 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.135059 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:20.281310 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.281285 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9"] Jul 30 21:02:20.283773 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:02:20.283744 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad83382_33cb_4dc1_a244_c85fbd8e4264.slice/crio-c59de71d6a2d984d1c28ff1faca7270affea7ddfe398c338dfc3d8d1027499e5 WatchSource:0}: Error finding container c59de71d6a2d984d1c28ff1faca7270affea7ddfe398c338dfc3d8d1027499e5: Status 404 returned error can't find the container with id c59de71d6a2d984d1c28ff1faca7270affea7ddfe398c338dfc3d8d1027499e5 Jul 30 21:02:20.285556 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:20.285540 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 21:02:21.278553 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.278502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" event={"ID":"7ad83382-33cb-4dc1-a244-c85fbd8e4264","Type":"ContainerStarted","Data":"c59de71d6a2d984d1c28ff1faca7270affea7ddfe398c338dfc3d8d1027499e5"} Jul 30 21:02:21.605639 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.605536 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt"] Jul 30 21:02:21.609978 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.609955 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.663929 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.663893 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 21:02:21.664254 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.664224 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 21:02:21.664395 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.664266 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dq824\"" Jul 30 21:02:21.664785 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.664764 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 21:02:21.664951 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.664930 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 21:02:21.667696 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.667671 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt"] Jul 30 21:02:21.730044 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.730008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-tmp\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.730220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.730063 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.730220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.730099 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-manifests\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.730220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.730129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv7mw\" (UniqueName: \"kubernetes.io/projected/f066bb5f-d3ee-44fd-973d-2d864a4e932e-kube-api-access-lv7mw\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.730334 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.730274 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831207 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-tmp\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831367 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831367 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831255 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-manifests\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831367 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831284 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv7mw\" (UniqueName: \"kubernetes.io/projected/f066bb5f-d3ee-44fd-973d-2d864a4e932e-kube-api-access-lv7mw\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831547 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.831766 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.831739 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-manifests\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.833815 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.833781 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f066bb5f-d3ee-44fd-973d-2d864a4e932e-tmp\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.834274 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.834251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.834369 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.834299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f066bb5f-d3ee-44fd-973d-2d864a4e932e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.839223 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.839204 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv7mw\" (UniqueName: \"kubernetes.io/projected/f066bb5f-d3ee-44fd-973d-2d864a4e932e-kube-api-access-lv7mw\") pod \"opendatahub-operator-controller-manager-5ddcd94768-klwvt\" (UID: \"f066bb5f-d3ee-44fd-973d-2d864a4e932e\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:21.929492 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:21.929400 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:22.214757 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:22.214733 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt"] Jul 30 21:02:22.216736 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:02:22.216713 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf066bb5f_d3ee_44fd_973d_2d864a4e932e.slice/crio-7f0e6cfe4c379cfbc765360e3d11817bcf8ed705ed936241a1d0939fa17675a6 WatchSource:0}: Error finding container 7f0e6cfe4c379cfbc765360e3d11817bcf8ed705ed936241a1d0939fa17675a6: Status 404 returned error can't find the container with id 7f0e6cfe4c379cfbc765360e3d11817bcf8ed705ed936241a1d0939fa17675a6 Jul 30 21:02:22.284823 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:22.284791 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" event={"ID":"7ad83382-33cb-4dc1-a244-c85fbd8e4264","Type":"ContainerStarted","Data":"dcd39239607d80d7a6ec411574452c0f23601efcfc84817715d2da2009d5ba26"} Jul 30 21:02:22.285272 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:22.284858 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:22.286073 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:22.286048 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" event={"ID":"f066bb5f-d3ee-44fd-973d-2d864a4e932e","Type":"ContainerStarted","Data":"7f0e6cfe4c379cfbc765360e3d11817bcf8ed705ed936241a1d0939fa17675a6"} Jul 30 21:02:22.302485 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:22.302430 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" podStartSLOduration=1.456947 podStartE2EDuration="3.302414328s" totalImagesPullingTime="1.845467328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:22.301938841 +0000 UTC m=+313.001074733" watchObservedRunningTime="2026-07-30 21:02:22.302414328 +0000 UTC m=+313.001550218" Jul 30 21:02:25.303033 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:25.303002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" event={"ID":"f066bb5f-d3ee-44fd-973d-2d864a4e932e","Type":"ContainerStarted","Data":"0c290db44825605dbe9c1a1fa7baf58476bfaeb871a16c5f982a175bcd2532e2"} Jul 30 21:02:27.313839 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:27.313804 2567 generic.go:356] "Generic (PLEG): container finished" podID="f066bb5f-d3ee-44fd-973d-2d864a4e932e" containerID="0c290db44825605dbe9c1a1fa7baf58476bfaeb871a16c5f982a175bcd2532e2" exitCode=0 Jul 30 21:02:27.314199 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:27.313861 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" event={"ID":"f066bb5f-d3ee-44fd-973d-2d864a4e932e","Type":"ContainerDied","Data":"0c290db44825605dbe9c1a1fa7baf58476bfaeb871a16c5f982a175bcd2532e2"} Jul 30 21:02:29.326432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:29.326397 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" event={"ID":"f066bb5f-d3ee-44fd-973d-2d864a4e932e","Type":"ContainerStarted","Data":"6d6a864cd96c5b61110e6728792d16519d9a6691b2ff35c7bb9c154e40fa9389"} Jul 30 21:02:29.326870 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:29.326751 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:02:29.348935 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:29.348890 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" podStartSLOduration=2.773548061 podStartE2EDuration="8.34887523s" totalImagesPullingTime="4.481443336s" totalInitContainerRuntime="1.093883833s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:02:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:29.347445546 +0000 UTC m=+320.046581436" watchObservedRunningTime="2026-07-30 21:02:29.34887523 +0000 UTC m=+320.048011121" Jul 30 21:02:33.294436 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:33.294406 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76d7f64c45-rvlb9" Jul 30 21:02:40.333415 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:02:40.333376 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5ddcd94768-klwvt" Jul 30 21:03:01.137381 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.137344 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn"] Jul 30 21:03:01.144384 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.144356 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.147519 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.147499 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 21:03:01.148789 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.148767 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 21:03:01.149005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.148987 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 21:03:01.151847 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.151824 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9p49l\"" Jul 30 21:03:01.177436 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.177408 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn"] Jul 30 21:03:01.252405 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252405 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252406 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252569 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252657 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252704 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252688 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.252942 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.252748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hsmz\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-kube-api-access-2hsmz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354063 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354292 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354292 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354411 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354411 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.354411 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.355378 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.355349 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.355550 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.354524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hsmz\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-kube-api-access-2hsmz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.355761 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.355698 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.355761 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.355736 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.355761 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.355748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.357253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.356072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.357253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.356496 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.357253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.357119 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.362160 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.358256 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.363988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.363967 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.371824 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.371795 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hsmz\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-kube-api-access-2hsmz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.371946 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.371876 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e7fa39-cf0c-4f79-9c79-a796758e63c2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn\" (UID: \"b2e7fa39-cf0c-4f79-9c79-a796758e63c2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.456233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.456137 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:01.611498 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.611472 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn"] Jul 30 21:03:01.613035 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:01.613007 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e7fa39_cf0c_4f79_9c79_a796758e63c2.slice/crio-dbb6ddb290aeb8976f346a48a0bc80d622e5863bae963343b6cad48b8deaefeb WatchSource:0}: Error finding container dbb6ddb290aeb8976f346a48a0bc80d622e5863bae963343b6cad48b8deaefeb: Status 404 returned error can't find the container with id dbb6ddb290aeb8976f346a48a0bc80d622e5863bae963343b6cad48b8deaefeb Jul 30 21:03:01.955929 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.955894 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9"] Jul 30 21:03:01.959597 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.959556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:01.962449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.962430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 21:03:01.962603 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.962431 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 21:03:01.962681 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.962661 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 21:03:01.963033 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.962963 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jgslc\"" Jul 30 21:03:01.975856 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:01.975833 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9"] Jul 30 21:03:02.063048 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063064 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063101 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063132 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.063221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.063199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfl2z\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-kube-api-access-pfl2z\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.164633 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164716 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164739 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfl2z\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-kube-api-access-pfl2z\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164776 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164835 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165051 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.164860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.165541 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.165519 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.166969 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.166949 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.167064 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.167025 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.167358 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.167344 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.167397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.167382 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.187934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.187916 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.189758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.189739 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfl2z\" (UniqueName: \"kubernetes.io/projected/6eb7c4d1-2cea-4b70-8775-0b2149ba16f2-kube-api-access-pfl2z\") pod \"istiod-openshift-gateway-5677ccb49b-9flg9\" (UID: \"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.272517 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.272495 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:02.431452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.431427 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9"] Jul 30 21:03:02.433455 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:02.433428 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb7c4d1_2cea_4b70_8775_0b2149ba16f2.slice/crio-d01ad2e9c0e8c5f295646b0baecf767dc3d7a62fa916d6a618246d066936d083 WatchSource:0}: Error finding container d01ad2e9c0e8c5f295646b0baecf767dc3d7a62fa916d6a618246d066936d083: Status 404 returned error can't find the container with id d01ad2e9c0e8c5f295646b0baecf767dc3d7a62fa916d6a618246d066936d083 Jul 30 21:03:02.480630 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.480599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" event={"ID":"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2","Type":"ContainerStarted","Data":"d01ad2e9c0e8c5f295646b0baecf767dc3d7a62fa916d6a618246d066936d083"} Jul 30 21:03:02.481734 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:02.481714 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" event={"ID":"b2e7fa39-cf0c-4f79-9c79-a796758e63c2","Type":"ContainerStarted","Data":"dbb6ddb290aeb8976f346a48a0bc80d622e5863bae963343b6cad48b8deaefeb"} Jul 30 21:03:06.374569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.374529 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:03:06.374957 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.374631 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:03:06.374957 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.374662 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:03:06.375667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.375639 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:03:06.375746 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.375693 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:03:06.509289 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.509244 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" event={"ID":"6eb7c4d1-2cea-4b70-8775-0b2149ba16f2","Type":"ContainerStarted","Data":"dd741fc932828f6fdf67a4f30753b0f39d99d5fed58e8dfad382d6a2f010d003"} Jul 30 21:03:06.509475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.509400 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:06.511253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.511221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" event={"ID":"b2e7fa39-cf0c-4f79-9c79-a796758e63c2","Type":"ContainerStarted","Data":"631185577d60d56cd61c84a4f0ad7bc139733f37c482b97206b05c5ec9078c27"} Jul 30 21:03:06.536436 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:06.536388 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" podStartSLOduration=1.596630312 podStartE2EDuration="5.536372713s" totalImagesPullingTime="3.939742401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:03:06.533555058 +0000 UTC m=+357.232690948" watchObservedRunningTime="2026-07-30 21:03:06.536372713 +0000 UTC m=+357.235508602" Jul 30 21:03:07.456340 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.456305 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:07.460893 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.460871 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:07.485323 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.485282 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" podStartSLOduration=1.726109761 podStartE2EDuration="6.485269174s" totalImagesPullingTime="4.759159413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:03:06.559989992 +0000 UTC m=+357.259125884" watchObservedRunningTime="2026-07-30 21:03:07.485269174 +0000 UTC m=+358.184405063" Jul 30 21:03:07.515698 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.515675 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:07.516511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.516490 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn" Jul 30 21:03:07.517071 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:07.517054 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9flg9" Jul 30 21:03:46.858888 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.858854 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895"] Jul 30 21:03:46.866397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.866376 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:46.868524 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.868505 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 30 21:03:46.869007 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.868991 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k4jhp\"" Jul 30 21:03:46.869103 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.868997 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 30 21:03:46.872749 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.872725 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895"] Jul 30 21:03:46.937387 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.937359 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:46.937526 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.937422 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b4qj\" (UniqueName: \"kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:46.937526 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.937493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:46.959951 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.959922 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv"] Jul 30 21:03:46.962300 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.962284 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:46.971723 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:46.971703 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv"] Jul 30 21:03:47.037920 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.037888 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.038066 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.037936 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.038066 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.037976 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvdgc\" (UniqueName: \"kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.038066 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.038015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b4qj\" (UniqueName: \"kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.038197 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.038170 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.038233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.038207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.038296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.038280 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.038493 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.038478 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.045715 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.045694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b4qj\" (UniqueName: \"kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.064687 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.064667 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp"] Jul 30 21:03:47.067275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.067261 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.075835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.075809 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp"] Jul 30 21:03:47.139398 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.139398 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139376 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvdgc\" (UniqueName: \"kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.139606 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139402 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.139606 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139428 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ml9j\" (UniqueName: \"kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.139606 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.139606 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139492 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.139783 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139750 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.139820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.139782 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.146679 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.146657 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvdgc\" (UniqueName: \"kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.159990 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.159968 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6"] Jul 30 21:03:47.162268 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.162253 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.171712 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.171691 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6"] Jul 30 21:03:47.177295 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.177277 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:47.240781 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.240752 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.240907 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.240795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ml9j\" (UniqueName: \"kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.240907 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.240869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.241029 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.240937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.241098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.241045 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.241307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.241097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f26nc\" (UniqueName: \"kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.241474 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.241453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.241745 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.241724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.248349 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.248330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ml9j\" (UniqueName: \"kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.272048 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.272023 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:47.314441 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.314417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895"] Jul 30 21:03:47.316667 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:47.316635 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeffc926_8125_4258_a4d8_f1ffeb751886.slice/crio-bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4 WatchSource:0}: Error finding container bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4: Status 404 returned error can't find the container with id bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4 Jul 30 21:03:47.342124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.342098 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.342244 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.342137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26nc\" (UniqueName: \"kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.342244 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.342190 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.342450 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.342431 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.342539 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.342469 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.349991 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.349967 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26nc\" (UniqueName: \"kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.377247 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.377223 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:47.411666 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.411626 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv"] Jul 30 21:03:47.413720 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:47.413696 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0636ea_0681_4a5b_a00c_ebffb362eab4.slice/crio-d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57 WatchSource:0}: Error finding container d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57: Status 404 returned error can't find the container with id d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57 Jul 30 21:03:47.472518 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.472495 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:47.516089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.516061 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp"] Jul 30 21:03:47.517185 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:47.517123 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33b48be_37f8_42c7_b9d0_4b3593ca0700.slice/crio-4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb WatchSource:0}: Error finding container 4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb: Status 404 returned error can't find the container with id 4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb Jul 30 21:03:47.613828 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.613804 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6"] Jul 30 21:03:47.615080 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:03:47.615054 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b335c7_ef67_4715_bb30_661d5c9ac267.slice/crio-89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7 WatchSource:0}: Error finding container 89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7: Status 404 returned error can't find the container with id 89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7 Jul 30 21:03:47.706927 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.706886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerStarted","Data":"08c7ee8e83aa747ec4a56a71314571a10090f13b84e8b34f94ad007e3ab1067e"} Jul 30 21:03:47.707091 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.706954 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerStarted","Data":"89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7"} Jul 30 21:03:47.708505 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.708477 2567 generic.go:356] "Generic (PLEG): container finished" podID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerID="6329c5d8f2b095b2696c43d20e5f6e5f04859196b3a5609a1d60007e44c6df37" exitCode=0 Jul 30 21:03:47.708637 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.708556 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" event={"ID":"d33b48be-37f8-42c7-b9d0-4b3593ca0700","Type":"ContainerDied","Data":"6329c5d8f2b095b2696c43d20e5f6e5f04859196b3a5609a1d60007e44c6df37"} Jul 30 21:03:47.708637 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.708623 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" event={"ID":"d33b48be-37f8-42c7-b9d0-4b3593ca0700","Type":"ContainerStarted","Data":"4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb"} Jul 30 21:03:47.710550 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.710530 2567 generic.go:356] "Generic (PLEG): container finished" podID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerID="26c9fd8a90cf9e2332413c135ac0ffc927d33cc34d674bfd87b953cdf05aaa2b" exitCode=0 Jul 30 21:03:47.710641 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.710605 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" event={"ID":"1f0636ea-0681-4a5b-a00c-ebffb362eab4","Type":"ContainerDied","Data":"26c9fd8a90cf9e2332413c135ac0ffc927d33cc34d674bfd87b953cdf05aaa2b"} Jul 30 21:03:47.710711 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.710674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" event={"ID":"1f0636ea-0681-4a5b-a00c-ebffb362eab4","Type":"ContainerStarted","Data":"d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57"} Jul 30 21:03:47.712451 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.712422 2567 generic.go:356] "Generic (PLEG): container finished" podID="feffc926-8125-4258-a4d8-f1ffeb751886" containerID="9884f5a7a29fcae6c4b16de757ef889544a813c15f971fb9498fe94e8c5dce0c" exitCode=0 Jul 30 21:03:47.712522 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.712444 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" event={"ID":"feffc926-8125-4258-a4d8-f1ffeb751886","Type":"ContainerDied","Data":"9884f5a7a29fcae6c4b16de757ef889544a813c15f971fb9498fe94e8c5dce0c"} Jul 30 21:03:47.712522 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:47.712487 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" event={"ID":"feffc926-8125-4258-a4d8-f1ffeb751886","Type":"ContainerStarted","Data":"bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4"} Jul 30 21:03:48.720113 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:48.720076 2567 generic.go:356] "Generic (PLEG): container finished" podID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerID="08c7ee8e83aa747ec4a56a71314571a10090f13b84e8b34f94ad007e3ab1067e" exitCode=0 Jul 30 21:03:48.720488 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:48.720161 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerDied","Data":"08c7ee8e83aa747ec4a56a71314571a10090f13b84e8b34f94ad007e3ab1067e"} Jul 30 21:03:48.721925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:48.721904 2567 generic.go:356] "Generic (PLEG): container finished" podID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerID="f79d99b79c672887fdaa912bc237fbfea9efa493b5fd3dbaecb9aaa10c6d602a" exitCode=0 Jul 30 21:03:48.722015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:48.721991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" event={"ID":"d33b48be-37f8-42c7-b9d0-4b3593ca0700","Type":"ContainerDied","Data":"f79d99b79c672887fdaa912bc237fbfea9efa493b5fd3dbaecb9aaa10c6d602a"} Jul 30 21:03:49.728705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:49.728676 2567 generic.go:356] "Generic (PLEG): container finished" podID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerID="2f4f27acca6bdf0af9faa411d2988b4170e47a768a6ec838de7e1ffdbd175d32" exitCode=0 Jul 30 21:03:49.729078 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:49.728753 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" event={"ID":"d33b48be-37f8-42c7-b9d0-4b3593ca0700","Type":"ContainerDied","Data":"2f4f27acca6bdf0af9faa411d2988b4170e47a768a6ec838de7e1ffdbd175d32"} Jul 30 21:03:50.735440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:50.735359 2567 generic.go:356] "Generic (PLEG): container finished" podID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerID="30a7eb6d18cf9075db5fda641a0b896ea1d67a84fb474d78d9cdfd4e8c374bd2" exitCode=0 Jul 30 21:03:50.735889 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:50.735445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerDied","Data":"30a7eb6d18cf9075db5fda641a0b896ea1d67a84fb474d78d9cdfd4e8c374bd2"} Jul 30 21:03:50.737382 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:50.737361 2567 generic.go:356] "Generic (PLEG): container finished" podID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerID="2f8ab49a11ac57c64525962b092786f99fe8e85bdcd0a41499064d1f77b1ae80" exitCode=0 Jul 30 21:03:50.737470 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:50.737447 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" event={"ID":"1f0636ea-0681-4a5b-a00c-ebffb362eab4","Type":"ContainerDied","Data":"2f8ab49a11ac57c64525962b092786f99fe8e85bdcd0a41499064d1f77b1ae80"} Jul 30 21:03:50.997591 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:50.997556 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:51.077243 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.077218 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ml9j\" (UniqueName: \"kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j\") pod \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " Jul 30 21:03:51.077330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.077307 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util\") pod \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " Jul 30 21:03:51.077365 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.077332 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle\") pod \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\" (UID: \"d33b48be-37f8-42c7-b9d0-4b3593ca0700\") " Jul 30 21:03:51.077970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.077938 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle" (OuterVolumeSpecName: "bundle") pod "d33b48be-37f8-42c7-b9d0-4b3593ca0700" (UID: "d33b48be-37f8-42c7-b9d0-4b3593ca0700"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:51.079150 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.079122 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j" (OuterVolumeSpecName: "kube-api-access-2ml9j") pod "d33b48be-37f8-42c7-b9d0-4b3593ca0700" (UID: "d33b48be-37f8-42c7-b9d0-4b3593ca0700"). InnerVolumeSpecName "kube-api-access-2ml9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:03:51.082748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.082726 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util" (OuterVolumeSpecName: "util") pod "d33b48be-37f8-42c7-b9d0-4b3593ca0700" (UID: "d33b48be-37f8-42c7-b9d0-4b3593ca0700"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:51.178816 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.178793 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:51.178816 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.178813 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d33b48be-37f8-42c7-b9d0-4b3593ca0700-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:51.178941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.178823 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ml9j\" (UniqueName: \"kubernetes.io/projected/d33b48be-37f8-42c7-b9d0-4b3593ca0700-kube-api-access-2ml9j\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:51.744074 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.743984 2567 generic.go:356] "Generic (PLEG): container finished" podID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerID="cbf96365d509f332d0eb41355838ccb7263bf0de03021b3efa6dff5fdb41b071" exitCode=0 Jul 30 21:03:51.744496 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.744069 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerDied","Data":"cbf96365d509f332d0eb41355838ccb7263bf0de03021b3efa6dff5fdb41b071"} Jul 30 21:03:51.745791 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.745775 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" Jul 30 21:03:51.745890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.745771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kksvp" event={"ID":"d33b48be-37f8-42c7-b9d0-4b3593ca0700","Type":"ContainerDied","Data":"4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb"} Jul 30 21:03:51.745890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.745886 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4148c8700261c84a2fb8782015541884873614c1efe1c1a4fe00ef738cdde8fb" Jul 30 21:03:51.747850 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.747827 2567 generic.go:356] "Generic (PLEG): container finished" podID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerID="929358cdaecccbb971f99a683c4f89d72f66a4b689c03fd668070104c5c4b4b2" exitCode=0 Jul 30 21:03:51.747956 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.747939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" event={"ID":"1f0636ea-0681-4a5b-a00c-ebffb362eab4","Type":"ContainerDied","Data":"929358cdaecccbb971f99a683c4f89d72f66a4b689c03fd668070104c5c4b4b2"} Jul 30 21:03:51.749750 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.749732 2567 generic.go:356] "Generic (PLEG): container finished" podID="feffc926-8125-4258-a4d8-f1ffeb751886" containerID="f51f5edbb4d669e8727e71b6363c38a4d1f18293845d6cdf3c047f01763a3bd9" exitCode=0 Jul 30 21:03:51.749832 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:51.749762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" event={"ID":"feffc926-8125-4258-a4d8-f1ffeb751886","Type":"ContainerDied","Data":"f51f5edbb4d669e8727e71b6363c38a4d1f18293845d6cdf3c047f01763a3bd9"} Jul 30 21:03:52.756658 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.756621 2567 generic.go:356] "Generic (PLEG): container finished" podID="feffc926-8125-4258-a4d8-f1ffeb751886" containerID="452a7a10f3e5554198eb3bdb2beefadefb60718b960a4dae118f38a018922fb4" exitCode=0 Jul 30 21:03:52.757121 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.756712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" event={"ID":"feffc926-8125-4258-a4d8-f1ffeb751886","Type":"ContainerDied","Data":"452a7a10f3e5554198eb3bdb2beefadefb60718b960a4dae118f38a018922fb4"} Jul 30 21:03:52.896280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.896256 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:52.926558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.926533 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:52.995541 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995514 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f26nc\" (UniqueName: \"kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc\") pod \"18b335c7-ef67-4715-bb30-661d5c9ac267\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " Jul 30 21:03:52.995716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995557 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvdgc\" (UniqueName: \"kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc\") pod \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " Jul 30 21:03:52.995716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995609 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle\") pod \"18b335c7-ef67-4715-bb30-661d5c9ac267\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " Jul 30 21:03:52.995716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995645 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle\") pod \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " Jul 30 21:03:52.995716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995703 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util\") pod \"18b335c7-ef67-4715-bb30-661d5c9ac267\" (UID: \"18b335c7-ef67-4715-bb30-661d5c9ac267\") " Jul 30 21:03:52.995880 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.995723 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util\") pod \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\" (UID: \"1f0636ea-0681-4a5b-a00c-ebffb362eab4\") " Jul 30 21:03:52.996253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.996226 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle" (OuterVolumeSpecName: "bundle") pod "1f0636ea-0681-4a5b-a00c-ebffb362eab4" (UID: "1f0636ea-0681-4a5b-a00c-ebffb362eab4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:52.996336 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.996231 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle" (OuterVolumeSpecName: "bundle") pod "18b335c7-ef67-4715-bb30-661d5c9ac267" (UID: "18b335c7-ef67-4715-bb30-661d5c9ac267"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:52.997522 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.997503 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc" (OuterVolumeSpecName: "kube-api-access-f26nc") pod "18b335c7-ef67-4715-bb30-661d5c9ac267" (UID: "18b335c7-ef67-4715-bb30-661d5c9ac267"). InnerVolumeSpecName "kube-api-access-f26nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:03:52.997968 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:52.997940 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc" (OuterVolumeSpecName: "kube-api-access-dvdgc") pod "1f0636ea-0681-4a5b-a00c-ebffb362eab4" (UID: "1f0636ea-0681-4a5b-a00c-ebffb362eab4"). InnerVolumeSpecName "kube-api-access-dvdgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:03:53.096667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.096611 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f26nc\" (UniqueName: \"kubernetes.io/projected/18b335c7-ef67-4715-bb30-661d5c9ac267-kube-api-access-f26nc\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.096667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.096638 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvdgc\" (UniqueName: \"kubernetes.io/projected/1f0636ea-0681-4a5b-a00c-ebffb362eab4-kube-api-access-dvdgc\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.096667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.096653 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.096826 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.096665 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.169717 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.169683 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util" (OuterVolumeSpecName: "util") pod "18b335c7-ef67-4715-bb30-661d5c9ac267" (UID: "18b335c7-ef67-4715-bb30-661d5c9ac267"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:53.170025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.170001 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util" (OuterVolumeSpecName: "util") pod "1f0636ea-0681-4a5b-a00c-ebffb362eab4" (UID: "1f0636ea-0681-4a5b-a00c-ebffb362eab4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:53.197848 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.197827 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18b335c7-ef67-4715-bb30-661d5c9ac267-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.197848 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.197846 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1f0636ea-0681-4a5b-a00c-ebffb362eab4-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:53.764038 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.764010 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" Jul 30 21:03:53.764476 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.764003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcmtx6" event={"ID":"18b335c7-ef67-4715-bb30-661d5c9ac267","Type":"ContainerDied","Data":"89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7"} Jul 30 21:03:53.764476 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.764138 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89e9ef2c0df1bf01401156b81cbae354e28c7e4a088dd5c92b4caaa682f2f8c7" Jul 30 21:03:53.765986 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.765967 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" Jul 30 21:03:53.766127 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.765977 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hrqbv" event={"ID":"1f0636ea-0681-4a5b-a00c-ebffb362eab4","Type":"ContainerDied","Data":"d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57"} Jul 30 21:03:53.766127 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.766023 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4b5f431f4b5cf1ac1f6c7409dba02a66c66b3bad848fc4f0e64329af3b89c57" Jul 30 21:03:53.896769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:53.896746 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:03:54.005773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.005742 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b4qj\" (UniqueName: \"kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj\") pod \"feffc926-8125-4258-a4d8-f1ffeb751886\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " Jul 30 21:03:54.005912 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.005801 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle\") pod \"feffc926-8125-4258-a4d8-f1ffeb751886\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " Jul 30 21:03:54.005912 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.005880 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util\") pod \"feffc926-8125-4258-a4d8-f1ffeb751886\" (UID: \"feffc926-8125-4258-a4d8-f1ffeb751886\") " Jul 30 21:03:54.006375 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.006346 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle" (OuterVolumeSpecName: "bundle") pod "feffc926-8125-4258-a4d8-f1ffeb751886" (UID: "feffc926-8125-4258-a4d8-f1ffeb751886"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:54.007795 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.007769 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj" (OuterVolumeSpecName: "kube-api-access-9b4qj") pod "feffc926-8125-4258-a4d8-f1ffeb751886" (UID: "feffc926-8125-4258-a4d8-f1ffeb751886"). InnerVolumeSpecName "kube-api-access-9b4qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:03:54.010516 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.010485 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util" (OuterVolumeSpecName: "util") pod "feffc926-8125-4258-a4d8-f1ffeb751886" (UID: "feffc926-8125-4258-a4d8-f1ffeb751886"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:03:54.106523 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.106448 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-util\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:54.106523 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.106486 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b4qj\" (UniqueName: \"kubernetes.io/projected/feffc926-8125-4258-a4d8-f1ffeb751886-kube-api-access-9b4qj\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:54.106523 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.106495 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/feffc926-8125-4258-a4d8-f1ffeb751886-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:03:54.772845 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.772799 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" event={"ID":"feffc926-8125-4258-a4d8-f1ffeb751886","Type":"ContainerDied","Data":"bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4"} Jul 30 21:03:54.773257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.772852 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bffd7a1b1b614d7bb76ab877525903167fb18315fb213d55752bf173509ccbf4" Jul 30 21:03:54.773257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:03:54.772924 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cc4895" Jul 30 21:04:18.688979 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.688949 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69d7ddf5f8-tr28w"] Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689300 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689313 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689326 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689331 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689338 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689344 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689356 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689362 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689372 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689378 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689388 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689393 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689399 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689404 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689409 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689414 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="util" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689420 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689424 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="pull" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689430 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689435 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689441 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689446 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="extract" Jul 30 21:04:18.689440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689451 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="extract" Jul 30 21:04:18.690112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689455 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="extract" Jul 30 21:04:18.690112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689504 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="feffc926-8125-4258-a4d8-f1ffeb751886" containerName="extract" Jul 30 21:04:18.690112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689514 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f0636ea-0681-4a5b-a00c-ebffb362eab4" containerName="extract" Jul 30 21:04:18.690112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689519 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33b48be-37f8-42c7-b9d0-4b3593ca0700" containerName="extract" Jul 30 21:04:18.690112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.689525 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b335c7-ef67-4715-bb30-661d5c9ac267" containerName="extract" Jul 30 21:04:18.692877 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.692858 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.706514 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.706487 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69d7ddf5f8-tr28w"] Jul 30 21:04:18.713699 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-service-ca\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.713820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-oauth-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.713820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713746 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nth5\" (UniqueName: \"kubernetes.io/projected/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-kube-api-access-5nth5\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.713913 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-oauth-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.713913 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713881 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.713913 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-trusted-ca-bundle\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.714025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.713974 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.814855 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814818 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.814988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814871 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-service-ca\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.814988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814906 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-oauth-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.814988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814932 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nth5\" (UniqueName: \"kubernetes.io/projected/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-kube-api-access-5nth5\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.814988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814958 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-oauth-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.814997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.815017 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-trusted-ca-bundle\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815565 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.815536 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.815664 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-oauth-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.815725 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-service-ca\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.815931 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.815911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-trusted-ca-bundle\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.817425 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.817406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-oauth-config\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.817622 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.817604 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-console-serving-cert\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:18.822917 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:18.822895 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nth5\" (UniqueName: \"kubernetes.io/projected/9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0-kube-api-access-5nth5\") pod \"console-69d7ddf5f8-tr28w\" (UID: \"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0\") " pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:19.010059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:19.010023 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:19.150061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:19.150038 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69d7ddf5f8-tr28w"] Jul 30 21:04:19.152145 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:04:19.152118 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a27b83b_8a40_45fa_bc0e_6e5329c3b0a0.slice/crio-ac9625c1e0f0ae2c3035643bb1033dab918070916a77ed7869df9ef511466ed0 WatchSource:0}: Error finding container ac9625c1e0f0ae2c3035643bb1033dab918070916a77ed7869df9ef511466ed0: Status 404 returned error can't find the container with id ac9625c1e0f0ae2c3035643bb1033dab918070916a77ed7869df9ef511466ed0 Jul 30 21:04:19.898679 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:19.898647 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d7ddf5f8-tr28w" event={"ID":"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0","Type":"ContainerStarted","Data":"0e2041806df21b47208caa9febfa081dc030ca097c5a23cb1bd9b5c28b564830"} Jul 30 21:04:19.899066 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:19.898705 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69d7ddf5f8-tr28w" event={"ID":"9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0","Type":"ContainerStarted","Data":"ac9625c1e0f0ae2c3035643bb1033dab918070916a77ed7869df9ef511466ed0"} Jul 30 21:04:19.926232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:19.926182 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69d7ddf5f8-tr28w" podStartSLOduration=1.926168018 podStartE2EDuration="1.926168018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:04:19.923493065 +0000 UTC m=+430.622628955" watchObservedRunningTime="2026-07-30 21:04:19.926168018 +0000 UTC m=+430.625303972" Jul 30 21:04:29.010976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:29.010942 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:29.011421 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:29.011031 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:29.015510 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:29.015488 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:29.953853 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:29.953823 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69d7ddf5f8-tr28w" Jul 30 21:04:30.002640 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:30.002608 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:04:55.024484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.024440 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c57b46c86-kztvb" podUID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" containerName="console" containerID="cri-o://41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4" gracePeriod=15 Jul 30 21:04:55.270963 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.270936 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:04:55.417506 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417431 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417506 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417467 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417524 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417544 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6ss7\" (UniqueName: \"kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417621 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417650 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.417758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.417686 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca\") pod \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\" (UID: \"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa\") " Jul 30 21:04:55.418061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.418035 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:04:55.418118 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.418096 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:04:55.418174 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.418146 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config" (OuterVolumeSpecName: "console-config") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:04:55.418282 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.418155 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca" (OuterVolumeSpecName: "service-ca") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:04:55.419723 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.419698 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:04:55.419835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.419739 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7" (OuterVolumeSpecName: "kube-api-access-j6ss7") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "kube-api-access-j6ss7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:04:55.419835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.419798 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" (UID: "99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:04:55.518933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518908 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-oauth-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.518933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518932 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.519082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518945 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-service-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.519082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518957 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-oauth-config\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.519082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518969 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-console-serving-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.519082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518981 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-trusted-ca-bundle\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.519082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.518993 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6ss7\" (UniqueName: \"kubernetes.io/projected/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa-kube-api-access-j6ss7\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:04:55.820676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.820649 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667"] Jul 30 21:04:55.821054 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.821030 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" containerName="console" Jul 30 21:04:55.821107 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.821058 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" containerName="console" Jul 30 21:04:55.821166 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.821155 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" containerName="console" Jul 30 21:04:55.824895 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.824873 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.827074 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.827045 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wjdv4\"" Jul 30 21:04:55.836626 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.836602 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667"] Jul 30 21:04:55.922594 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.922790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00fcefcc-2f59-405f-96a3-da5265867814-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.922790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922660 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bktgs\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-kube-api-access-bktgs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.922790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00fcefcc-2f59-405f-96a3-da5265867814-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.922790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.923014 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922817 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.923014 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922862 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.923014 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922917 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:55.923014 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:55.922970 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024017 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.023948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024017 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.023982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024017 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024017 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024286 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024035 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00fcefcc-2f59-405f-96a3-da5265867814-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024286 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024059 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bktgs\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-kube-api-access-bktgs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024286 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024090 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00fcefcc-2f59-405f-96a3-da5265867814-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024286 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024122 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024349 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024404 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024484 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024440 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024518 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024761 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.024934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024772 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.025031 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.024950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00fcefcc-2f59-405f-96a3-da5265867814-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.026379 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.026352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00fcefcc-2f59-405f-96a3-da5265867814-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.026686 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.026666 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00fcefcc-2f59-405f-96a3-da5265867814-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.043052 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.043020 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.043565 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.043543 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bktgs\" (UniqueName: \"kubernetes.io/projected/00fcefcc-2f59-405f-96a3-da5265867814-kube-api-access-bktgs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-s8667\" (UID: \"00fcefcc-2f59-405f-96a3-da5265867814\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.090961 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.090934 2567 generic.go:356] "Generic (PLEG): container finished" podID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" containerID="41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4" exitCode=0 Jul 30 21:04:56.091089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.090978 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57b46c86-kztvb" event={"ID":"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa","Type":"ContainerDied","Data":"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4"} Jul 30 21:04:56.091089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.090990 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57b46c86-kztvb" Jul 30 21:04:56.091089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.091036 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57b46c86-kztvb" event={"ID":"99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa","Type":"ContainerDied","Data":"07dbeb21b572fa83feaca1173d14a8e36ac6c8b158f5b88e75674fee9022d152"} Jul 30 21:04:56.091089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.091056 2567 scope.go:117] "RemoveContainer" containerID="41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4" Jul 30 21:04:56.101734 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.101717 2567 scope.go:117] "RemoveContainer" containerID="41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4" Jul 30 21:04:56.102011 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:04:56.101995 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4\": container with ID starting with 41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4 not found: ID does not exist" containerID="41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4" Jul 30 21:04:56.102075 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.102016 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4"} err="failed to get container status \"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4\": rpc error: code = NotFound desc = could not find container \"41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4\": container with ID starting with 41ba197c12675b3fd2206e63d68f6f7a91f052217b480ecbb3f091e1f4d7ddc4 not found: ID does not exist" Jul 30 21:04:56.114163 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.114136 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:04:56.119540 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.119517 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c57b46c86-kztvb"] Jul 30 21:04:56.139118 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.139096 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:56.273143 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.273114 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667"] Jul 30 21:04:56.274969 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:04:56.274912 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00fcefcc_2f59_405f_96a3_da5265867814.slice/crio-524f88e5dc8fe52eded8028b68594a460b182b62d9d91457be01d85e654a5ed3 WatchSource:0}: Error finding container 524f88e5dc8fe52eded8028b68594a460b182b62d9d91457be01d85e654a5ed3: Status 404 returned error can't find the container with id 524f88e5dc8fe52eded8028b68594a460b182b62d9d91457be01d85e654a5ed3 Jul 30 21:04:56.276996 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.276961 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:04:56.277103 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.277047 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:04:56.277103 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:56.277094 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:04:57.097910 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.097878 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" event={"ID":"00fcefcc-2f59-405f-96a3-da5265867814","Type":"ContainerStarted","Data":"15da38da85f93f3860ea1740c2fc7d0eb03b0be228b94fd3a23acff55306d7c1"} Jul 30 21:04:57.098330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.097939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" event={"ID":"00fcefcc-2f59-405f-96a3-da5265867814","Type":"ContainerStarted","Data":"524f88e5dc8fe52eded8028b68594a460b182b62d9d91457be01d85e654a5ed3"} Jul 30 21:04:57.119971 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.119918 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podStartSLOduration=2.119903283 podStartE2EDuration="2.119903283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:04:57.11739813 +0000 UTC m=+467.816534020" watchObservedRunningTime="2026-07-30 21:04:57.119903283 +0000 UTC m=+467.819039172" Jul 30 21:04:57.139872 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.139846 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:57.144534 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.144515 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:57.757721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:57.757688 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa" path="/var/lib/kubelet/pods/99513a5b-f2bc-4c59-9e7d-0da56f1bd2fa/volumes" Jul 30 21:04:58.104251 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:58.104179 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:04:58.105093 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:04:58.105075 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:06:16.348883 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.348808 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2"] Jul 30 21:06:16.351398 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.351379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.353898 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.353875 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 30 21:06:16.354015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.353923 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 30 21:06:16.354015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.353952 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 21:06:16.354015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.353978 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58scw\"" Jul 30 21:06:16.365137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.365111 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2"] Jul 30 21:06:16.396236 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.396208 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-maas-api-tls\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.396339 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.396256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kzn\" (UniqueName: \"kubernetes.io/projected/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-kube-api-access-c7kzn\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.396339 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.396292 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-openshift-service-ca\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.497326 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.497297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-maas-api-tls\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.497461 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.497340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kzn\" (UniqueName: \"kubernetes.io/projected/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-kube-api-access-c7kzn\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.497513 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.497458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-openshift-service-ca\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.498059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.498037 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-openshift-service-ca\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.499642 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.499623 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-maas-api-tls\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.504511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.504489 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kzn\" (UniqueName: \"kubernetes.io/projected/5cac8065-969f-4f4c-846d-0eb47ff4f7fe-kube-api-access-c7kzn\") pod \"maas-api-9f684b49f-4d2p2\" (UID: \"5cac8065-969f-4f4c-846d-0eb47ff4f7fe\") " pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.663617 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.663529 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:16.796179 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.796154 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2"] Jul 30 21:06:16.798044 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:06:16.798011 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cac8065_969f_4f4c_846d_0eb47ff4f7fe.slice/crio-bf272f5af2462622a53e8c8661af400b7411eed2d585b81f6373d8811bf55511 WatchSource:0}: Error finding container bf272f5af2462622a53e8c8661af400b7411eed2d585b81f6373d8811bf55511: Status 404 returned error can't find the container with id bf272f5af2462622a53e8c8661af400b7411eed2d585b81f6373d8811bf55511 Jul 30 21:06:16.832762 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.832738 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr"] Jul 30 21:06:16.835989 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.835972 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:16.838030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.838007 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 30 21:06:16.846451 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.846425 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr"] Jul 30 21:06:16.900672 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.900649 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592520a6-e0a9-4634-8366-53514a34e060-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:16.900791 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:16.900726 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99f6b\" (UniqueName: \"kubernetes.io/projected/592520a6-e0a9-4634-8366-53514a34e060-kube-api-access-99f6b\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.002018 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.001990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99f6b\" (UniqueName: \"kubernetes.io/projected/592520a6-e0a9-4634-8366-53514a34e060-kube-api-access-99f6b\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.002161 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.002046 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592520a6-e0a9-4634-8366-53514a34e060-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.002700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.002682 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592520a6-e0a9-4634-8366-53514a34e060-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.010067 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.010049 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99f6b\" (UniqueName: \"kubernetes.io/projected/592520a6-e0a9-4634-8366-53514a34e060-kube-api-access-99f6b\") pod \"payload-pre-processing-687dbb6cf6-w9btr\" (UID: \"592520a6-e0a9-4634-8366-53514a34e060\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.147052 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.147016 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:17.310705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.308758 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr"] Jul 30 21:06:17.320708 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:06:17.317950 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592520a6_e0a9_4634_8366_53514a34e060.slice/crio-0b48e399751a2fbe802030aa1e954c9b7360d8e7d476371052b37a261e352261 WatchSource:0}: Error finding container 0b48e399751a2fbe802030aa1e954c9b7360d8e7d476371052b37a261e352261: Status 404 returned error can't find the container with id 0b48e399751a2fbe802030aa1e954c9b7360d8e7d476371052b37a261e352261 Jul 30 21:06:17.529636 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.529597 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" event={"ID":"592520a6-e0a9-4634-8366-53514a34e060","Type":"ContainerStarted","Data":"0b48e399751a2fbe802030aa1e954c9b7360d8e7d476371052b37a261e352261"} Jul 30 21:06:17.530872 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:17.530848 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" event={"ID":"5cac8065-969f-4f4c-846d-0eb47ff4f7fe","Type":"ContainerStarted","Data":"bf272f5af2462622a53e8c8661af400b7411eed2d585b81f6373d8811bf55511"} Jul 30 21:06:21.560652 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.560607 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" event={"ID":"5cac8065-969f-4f4c-846d-0eb47ff4f7fe","Type":"ContainerStarted","Data":"88fb86617db438888b47cd61e6199210c1b7a22c925dca418dce57054ec65136"} Jul 30 21:06:21.561107 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.560718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:21.562086 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.562062 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" event={"ID":"592520a6-e0a9-4634-8366-53514a34e060","Type":"ContainerStarted","Data":"4bd10430bbcde99ffd91e85f6bf773171185a9ed35ec5a22d81c822aa0800610"} Jul 30 21:06:21.562206 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.562191 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:21.578824 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.578783 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" podStartSLOduration=1.838996976 podStartE2EDuration="5.578769702s" totalImagesPullingTime="3.739772726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:06:21.577359214 +0000 UTC m=+552.276495104" watchObservedRunningTime="2026-07-30 21:06:21.578769702 +0000 UTC m=+552.277905591" Jul 30 21:06:21.596400 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:21.596356 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" podStartSLOduration=2.374716124 podStartE2EDuration="5.596337876s" totalImagesPullingTime="3.221621752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:06:21.594059243 +0000 UTC m=+552.293195135" watchObservedRunningTime="2026-07-30 21:06:21.596337876 +0000 UTC m=+552.295473768" Jul 30 21:06:26.591660 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.591625 2567 generic.go:356] "Generic (PLEG): container finished" podID="00fcefcc-2f59-405f-96a3-da5265867814" containerID="15da38da85f93f3860ea1740c2fc7d0eb03b0be228b94fd3a23acff55306d7c1" exitCode=0 Jul 30 21:06:26.592083 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.591703 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" event={"ID":"00fcefcc-2f59-405f-96a3-da5265867814","Type":"ContainerDied","Data":"15da38da85f93f3860ea1740c2fc7d0eb03b0be228b94fd3a23acff55306d7c1"} Jul 30 21:06:26.592083 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.592061 2567 scope.go:117] "RemoveContainer" containerID="15da38da85f93f3860ea1740c2fc7d0eb03b0be228b94fd3a23acff55306d7c1" Jul 30 21:06:26.592971 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.592945 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:06:26.593031 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.593010 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:06:26.593082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:26.593051 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:06:27.573512 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:27.573478 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-9f684b49f-4d2p2" Jul 30 21:06:27.603110 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:27.602258 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" event={"ID":"00fcefcc-2f59-405f-96a3-da5265867814","Type":"ContainerStarted","Data":"0ab66888bfebd6d9a557a36d0638890dd1ba3aab0744b6bdc82755b6894bfd31"} Jul 30 21:06:28.104959 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:28.104902 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:06:28.139302 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:28.139269 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:06:28.140369 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:28.140345 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8667 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 30 21:06:28.140464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:28.140393 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podUID="00fcefcc-2f59-405f-96a3-da5265867814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 30 21:06:29.140042 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:29.140003 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8667 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 30 21:06:29.140446 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:29.140061 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podUID="00fcefcc-2f59-405f-96a3-da5265867814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 30 21:06:30.140024 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:30.139986 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8667 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 30 21:06:30.140201 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:30.140043 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podUID="00fcefcc-2f59-405f-96a3-da5265867814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 30 21:06:31.139408 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:31.139371 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8667 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 30 21:06:31.139616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:31.139432 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podUID="00fcefcc-2f59-405f-96a3-da5265867814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 30 21:06:32.140328 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:32.140293 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-s8667 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 30 21:06:32.140743 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:32.140347 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" podUID="00fcefcc-2f59-405f-96a3-da5265867814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 30 21:06:32.570757 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:32.570727 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-w9btr" Jul 30 21:06:33.143440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:33.143411 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:06:33.144377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:06:33.144356 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-s8667" Jul 30 21:07:09.713342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:09.713310 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:07:09.714418 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:09.714401 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:07:14.943865 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.943822 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6"] Jul 30 21:07:14.946622 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.946605 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:14.948693 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.948671 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9gzdm\"" Jul 30 21:07:14.948817 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.948697 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 21:07:14.948817 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.948727 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 30 21:07:14.948817 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.948732 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 21:07:14.957657 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:14.957626 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6"] Jul 30 21:07:15.114556 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.114773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.114773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.114773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114704 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.114773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114731 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.114978 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.114809 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r7h9\" (UniqueName: \"kubernetes.io/projected/5ecebb9f-b6be-41ff-94ed-2088dca250bc-kube-api-access-5r7h9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216288 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216288 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216238 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216288 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216255 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216288 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r7h9\" (UniqueName: \"kubernetes.io/projected/5ecebb9f-b6be-41ff-94ed-2088dca250bc-kube-api-access-5r7h9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216648 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216320 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216648 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216350 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216765 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216707 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216765 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216715 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.216836 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.216782 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.218633 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.218602 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecebb9f-b6be-41ff-94ed-2088dca250bc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.218877 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.218860 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecebb9f-b6be-41ff-94ed-2088dca250bc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.223695 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.223677 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r7h9\" (UniqueName: \"kubernetes.io/projected/5ecebb9f-b6be-41ff-94ed-2088dca250bc-kube-api-access-5r7h9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dg9q6\" (UID: \"5ecebb9f-b6be-41ff-94ed-2088dca250bc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.257331 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.257307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:15.387784 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.387760 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6"] Jul 30 21:07:15.389609 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:07:15.389565 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecebb9f_b6be_41ff_94ed_2088dca250bc.slice/crio-3900bc5c745d5867e56e2c07a94168ec3fcc07f54801385ddc542b8e0cbd9357 WatchSource:0}: Error finding container 3900bc5c745d5867e56e2c07a94168ec3fcc07f54801385ddc542b8e0cbd9357: Status 404 returned error can't find the container with id 3900bc5c745d5867e56e2c07a94168ec3fcc07f54801385ddc542b8e0cbd9357 Jul 30 21:07:15.875388 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:15.875346 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" event={"ID":"5ecebb9f-b6be-41ff-94ed-2088dca250bc","Type":"ContainerStarted","Data":"3900bc5c745d5867e56e2c07a94168ec3fcc07f54801385ddc542b8e0cbd9357"} Jul 30 21:07:16.883063 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:16.883022 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" event={"ID":"5ecebb9f-b6be-41ff-94ed-2088dca250bc","Type":"ContainerStarted","Data":"1747381694f2135ecd014510282e79918a5bddc2c3c8126a94f4055e6493e656"} Jul 30 21:07:16.901621 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:16.901544 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" podStartSLOduration=1.873122569 podStartE2EDuration="2.901529324s" totalImagesPullingTime="1.028406755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:07:16.899956966 +0000 UTC m=+607.599092848" watchObservedRunningTime="2026-07-30 21:07:16.901529324 +0000 UTC m=+607.600665213" Jul 30 21:07:25.258102 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:25.258067 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:25.258499 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:25.258123 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:25.270602 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:25.270555 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:07:25.943954 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:07:25.943923 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dg9q6" Jul 30 21:12:09.748737 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:12:09.748660 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:12:09.751191 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:12:09.751164 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:17:09.786753 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:17:09.786724 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:17:09.792896 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:17:09.792872 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:22:09.822391 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:22:09.822286 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:22:09.831160 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:22:09.831140 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:26:18.105220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.105185 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:26:18.109137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.109117 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.111487 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.111467 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1b7f48d9-openshift-default-dockercfg-tk84q\"" Jul 30 21:26:18.125800 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.125778 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:26:18.178923 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.178896 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78gvf\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179033 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.178930 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179078 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179114 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179114 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179101 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179176 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179132 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179176 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179159 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179241 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.179241 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.179217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.279867 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.279833 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.279867 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.279869 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.279886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.279903 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.279918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78gvf\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280455 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280356 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280455 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280422 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280605 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280477 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280674 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280620 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.280972 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.280951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.282571 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.282541 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.282702 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.282546 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.288488 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.288466 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.288864 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.288840 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78gvf\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf\") pod \"e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.419398 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.419314 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:18.553665 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.553635 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:26:18.555003 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:26:18.554974 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d13aea_1b1a_41bb_a4b2_e7cb6b24ea1e.slice/crio-047fb332d42123968d821f88a810d18be17119bd01474363ce70b5941bc09a0a WatchSource:0}: Error finding container 047fb332d42123968d821f88a810d18be17119bd01474363ce70b5941bc09a0a: Status 404 returned error can't find the container with id 047fb332d42123968d821f88a810d18be17119bd01474363ce70b5941bc09a0a Jul 30 21:26:18.557083 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.557043 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:18.557178 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.557127 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:18.557178 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.557166 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:18.861482 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.861424 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerStarted","Data":"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82"} Jul 30 21:26:18.861678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.861508 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerStarted","Data":"047fb332d42123968d821f88a810d18be17119bd01474363ce70b5941bc09a0a"} Jul 30 21:26:18.883192 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:18.883134 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podStartSLOduration=0.88311462 podStartE2EDuration="883.11462ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:26:18.881879364 +0000 UTC m=+1749.581015255" watchObservedRunningTime="2026-07-30 21:26:18.88311462 +0000 UTC m=+1749.582250510" Jul 30 21:26:19.420379 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:19.420345 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:19.425126 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:19.425097 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:19.866172 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:19.866148 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:19.867083 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:19.867061 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:21.386098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.386061 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:26:21.389758 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.389740 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.392134 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.392111 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1b7f48d9-serving-cert\"" Jul 30 21:26:21.403457 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.403434 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:26:21.510998 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.510961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.510998 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.511004 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxx5q\" (UniqueName: \"kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.511210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.511080 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.612445 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.612414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxx5q\" (UniqueName: \"kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.612647 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.612507 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.612647 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.612623 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.613250 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.613230 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.615019 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.614992 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.621287 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.621267 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxx5q\" (UniqueName: \"kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q\") pod \"maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.700773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.700717 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:21.834692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.834663 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:26:21.836090 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:26:21.836067 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61e0dc3_35b9_4aea_8159_12986f06d8ef.slice/crio-102925bb71d75f75a82cc692cbb3d30e18047796d40fdfec8ca81caf0ffe195a WatchSource:0}: Error finding container 102925bb71d75f75a82cc692cbb3d30e18047796d40fdfec8ca81caf0ffe195a: Status 404 returned error can't find the container with id 102925bb71d75f75a82cc692cbb3d30e18047796d40fdfec8ca81caf0ffe195a Jul 30 21:26:21.837411 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.837392 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 21:26:21.880223 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:21.880193 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" event={"ID":"d61e0dc3-35b9-4aea-8159-12986f06d8ef","Type":"ContainerStarted","Data":"102925bb71d75f75a82cc692cbb3d30e18047796d40fdfec8ca81caf0ffe195a"} Jul 30 21:26:22.887397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:22.887362 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" event={"ID":"d61e0dc3-35b9-4aea-8159-12986f06d8ef","Type":"ContainerStarted","Data":"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6"} Jul 30 21:26:22.887783 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:22.887443 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:22.905269 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:22.905225 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" podStartSLOduration=1.394979124 podStartE2EDuration="1.905213048s" totalImagesPullingTime="510.233924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:26:22.902271977 +0000 UTC m=+1753.601407866" watchObservedRunningTime="2026-07-30 21:26:22.905213048 +0000 UTC m=+1753.604348962" Jul 30 21:26:28.906883 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:28.906843 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:26:34.962080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:34.962007 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:60188->10.133.0.44:15021: read: connection reset by peer" start-of-body= Jul 30 21:26:34.962523 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:34.962081 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": read tcp 10.133.0.2:60188->10.133.0.44:15021: read: connection reset by peer" Jul 30 21:26:34.982387 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:26:34.982360 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@1-19169-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d13aea_1b1a_41bb_a4b2_e7cb6b24ea1e.slice/crio-c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82.scope\": RecentStats: unable to find data in memory cache]" Jul 30 21:26:35.962715 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.962682 2567 generic.go:356] "Generic (PLEG): container finished" podID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerID="c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82" exitCode=0 Jul 30 21:26:35.963099 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.962764 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerDied","Data":"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82"} Jul 30 21:26:35.963187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.963172 2567 scope.go:117] "RemoveContainer" containerID="c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82" Jul 30 21:26:35.964185 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.964157 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:35.964245 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.964234 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:35.964290 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:35.964272 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:26:36.970290 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:36.970254 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerStarted","Data":"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0"} Jul 30 21:26:37.420254 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:37.420221 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:37.420489 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:37.420469 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:37.420550 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:37.420515 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:38.420378 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:38.420341 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:38.420821 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:38.420408 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:39.419843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:39.419811 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:39.420018 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:39.419863 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:40.419965 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:40.419923 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:40.420360 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:40.419981 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:41.419988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:41.419950 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:41.420378 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:41.420011 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:42.420444 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:42.420406 2567 patch_prober.go:28] interesting pod/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 30 21:26:42.420851 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:42.420462 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 30 21:26:43.424030 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:43.423997 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:43.424485 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:43.424452 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:26:43.425116 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:26:43.425091 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:27:02.257596 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.257497 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:27:02.258153 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.257759 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" podUID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" containerName="maas-api" containerID="cri-o://ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6" gracePeriod=30 Jul 30 21:27:02.514373 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.514303 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:27:02.605998 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.605936 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls\") pod \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " Jul 30 21:27:02.605998 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.605990 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxx5q\" (UniqueName: \"kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q\") pod \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " Jul 30 21:27:02.606247 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.606087 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca\") pod \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\" (UID: \"d61e0dc3-35b9-4aea-8159-12986f06d8ef\") " Jul 30 21:27:02.606753 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.606723 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d61e0dc3-35b9-4aea-8159-12986f06d8ef" (UID: "d61e0dc3-35b9-4aea-8159-12986f06d8ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:27:02.623663 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.623620 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d61e0dc3-35b9-4aea-8159-12986f06d8ef" (UID: "d61e0dc3-35b9-4aea-8159-12986f06d8ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 21:27:02.623858 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.623633 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q" (OuterVolumeSpecName: "kube-api-access-rxx5q") pod "d61e0dc3-35b9-4aea-8159-12986f06d8ef" (UID: "d61e0dc3-35b9-4aea-8159-12986f06d8ef"). InnerVolumeSpecName "kube-api-access-rxx5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:27:02.626382 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.626350 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:27:02.626689 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.626662 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" containerID="cri-o://f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0" gracePeriod=30 Jul 30 21:27:02.706900 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.706862 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61e0dc3-35b9-4aea-8159-12986f06d8ef-openshift-service-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:02.706900 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.706892 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d61e0dc3-35b9-4aea-8159-12986f06d8ef-maas-api-tls\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:02.706900 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:02.706903 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxx5q\" (UniqueName: \"kubernetes.io/projected/d61e0dc3-35b9-4aea-8159-12986f06d8ef-kube-api-access-rxx5q\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:03.113368 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.113330 2567 generic.go:356] "Generic (PLEG): container finished" podID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" containerID="ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6" exitCode=0 Jul 30 21:27:03.113555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.113374 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" Jul 30 21:27:03.113555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.113392 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" event={"ID":"d61e0dc3-35b9-4aea-8159-12986f06d8ef","Type":"ContainerDied","Data":"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6"} Jul 30 21:27:03.113555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.113460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq" event={"ID":"d61e0dc3-35b9-4aea-8159-12986f06d8ef","Type":"ContainerDied","Data":"102925bb71d75f75a82cc692cbb3d30e18047796d40fdfec8ca81caf0ffe195a"} Jul 30 21:27:03.113555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.113486 2567 scope.go:117] "RemoveContainer" containerID="ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6" Jul 30 21:27:03.123464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.123447 2567 scope.go:117] "RemoveContainer" containerID="ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6" Jul 30 21:27:03.123736 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:27:03.123716 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6\": container with ID starting with ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6 not found: ID does not exist" containerID="ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6" Jul 30 21:27:03.123798 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.123744 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6"} err="failed to get container status \"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6\": rpc error: code = NotFound desc = could not find container \"ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6\": container with ID starting with ab9c6bdba4ab4ec5034a00cacdca24015416bf66672b46c9da7ffcab357ddfc6 not found: ID does not exist" Jul 30 21:27:03.143500 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.143473 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:27:03.154757 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.154729 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b7f48d9-6b97bdf677-zxkxq"] Jul 30 21:27:03.717722 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.717690 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:03.718244 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.718222 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" containerName="maas-api" Jul 30 21:27:03.718336 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.718247 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" containerName="maas-api" Jul 30 21:27:03.718384 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.718373 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" containerName="maas-api" Jul 30 21:27:03.721281 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.721261 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.723322 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.723303 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-60d76360-gw-openshift-default-dockercfg-llhg6\"" Jul 30 21:27:03.734372 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.734346 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:03.761387 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.761355 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d61e0dc3-35b9-4aea-8159-12986f06d8ef" path="/var/lib/kubelet/pods/d61e0dc3-35b9-4aea-8159-12986f06d8ef/volumes" Jul 30 21:27:03.816172 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.816133 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.816479 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.816459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.816721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.816689 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.816885 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.816870 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.817017 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.817002 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.817167 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.817152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.817616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.817595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.817842 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.817826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4nhp\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.818279 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.818259 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.919665 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.919878 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.919878 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919707 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.919878 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919831 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920050 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4nhp\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920050 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.919998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920050 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920214 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920078 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920214 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920462 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920436 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920622 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920527 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920859 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920835 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920946 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.920946 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.920854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.922243 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.922224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.922430 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.922415 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.928781 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.928758 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:03.928892 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:03.928831 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4nhp\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp\") pod \"e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:04.034336 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:04.034302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:04.175368 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:04.175339 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:04.176759 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:27:04.176731 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d14e8db_de3b_49f1_a5f9_0d34b189b6d1.slice/crio-3e9f3e9762b61ac0977b4407861cc6fbb2e2750a38038cec7f12058a37fc1637 WatchSource:0}: Error finding container 3e9f3e9762b61ac0977b4407861cc6fbb2e2750a38038cec7f12058a37fc1637: Status 404 returned error can't find the container with id 3e9f3e9762b61ac0977b4407861cc6fbb2e2750a38038cec7f12058a37fc1637 Jul 30 21:27:04.179219 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:04.179175 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:04.179289 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:04.179267 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:04.179337 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:04.179308 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:05.130268 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:05.130228 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerStarted","Data":"15f1ae3257a2e5f24700977a1d0ca4f15af99c823e5be9305782b7f7bd759ddd"} Jul 30 21:27:05.130676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:05.130313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerStarted","Data":"3e9f3e9762b61ac0977b4407861cc6fbb2e2750a38038cec7f12058a37fc1637"} Jul 30 21:27:05.150133 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:05.150083 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podStartSLOduration=2.150069225 podStartE2EDuration="2.150069225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:27:05.149087983 +0000 UTC m=+1795.848223872" watchObservedRunningTime="2026-07-30 21:27:05.150069225 +0000 UTC m=+1795.849205115" Jul 30 21:27:06.034588 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:06.034546 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:06.039287 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:06.039261 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:06.136274 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:06.136234 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:06.137188 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:06.137169 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:07.881986 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.881961 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:27:07.958985 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.958900 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.958985 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.958943 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.958985 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.958967 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78gvf\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959023 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959046 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959145 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959184 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959201 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:07.959235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959215 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959559 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959277 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data\") pod \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\" (UID: \"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e\") " Jul 30 21:27:07.959559 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959383 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:07.959559 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959544 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:07.959740 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959625 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:27:07.959810 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959786 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:07.959911 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959876 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:07.959947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959918 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:07.959947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959932 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:07.960025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.959947 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:07.961345 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.961322 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf" (OuterVolumeSpecName: "kube-api-access-78gvf") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "kube-api-access-78gvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:27:07.961451 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.961322 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:27:07.961569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.961546 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:27:07.961569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:07.961556 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" (UID: "14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:08.060975 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.060946 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:08.060975 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.060973 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78gvf\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-kube-api-access-78gvf\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:08.060975 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.060983 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:08.061200 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.060992 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:08.061200 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.061000 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:08.148719 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.148686 2567 generic.go:356] "Generic (PLEG): container finished" podID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerID="f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0" exitCode=0 Jul 30 21:27:08.148719 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.148729 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" Jul 30 21:27:08.148939 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.148770 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerDied","Data":"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0"} Jul 30 21:27:08.148939 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.148827 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9" event={"ID":"14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e","Type":"ContainerDied","Data":"047fb332d42123968d821f88a810d18be17119bd01474363ce70b5941bc09a0a"} Jul 30 21:27:08.148939 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.148851 2567 scope.go:117] "RemoveContainer" containerID="f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0" Jul 30 21:27:08.158338 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.158322 2567 scope.go:117] "RemoveContainer" containerID="c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82" Jul 30 21:27:08.166407 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.166390 2567 scope.go:117] "RemoveContainer" containerID="f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0" Jul 30 21:27:08.166708 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:27:08.166688 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0\": container with ID starting with f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0 not found: ID does not exist" containerID="f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0" Jul 30 21:27:08.166779 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.166717 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0"} err="failed to get container status \"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0\": rpc error: code = NotFound desc = could not find container \"f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0\": container with ID starting with f85f51d27bb24544105bb50c6d4d7f11752399fb7dc1727c7ab0100424bef1b0 not found: ID does not exist" Jul 30 21:27:08.166779 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.166734 2567 scope.go:117] "RemoveContainer" containerID="c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82" Jul 30 21:27:08.167150 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:27:08.167125 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82\": container with ID starting with c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82 not found: ID does not exist" containerID="c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82" Jul 30 21:27:08.167240 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.167159 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82"} err="failed to get container status \"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82\": rpc error: code = NotFound desc = could not find container \"c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82\": container with ID starting with c6f6fb1ee449906acd618d75d4c3b87ace2e9a3bb0a783df6334877acbb9ba82 not found: ID does not exist" Jul 30 21:27:08.170462 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.170440 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:27:08.175670 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:08.175650 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1b7f48d9-openshift-default-8bc8c5c7d-n8rv9"] Jul 30 21:27:09.758972 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:09.758936 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" path="/var/lib/kubelet/pods/14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e/volumes" Jul 30 21:27:09.859598 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:09.859550 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:27:09.868543 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:09.868519 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:27:20.217266 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.217234 2567 generic.go:356] "Generic (PLEG): container finished" podID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerID="15f1ae3257a2e5f24700977a1d0ca4f15af99c823e5be9305782b7f7bd759ddd" exitCode=0 Jul 30 21:27:20.217706 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.217308 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerDied","Data":"15f1ae3257a2e5f24700977a1d0ca4f15af99c823e5be9305782b7f7bd759ddd"} Jul 30 21:27:20.217759 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.217741 2567 scope.go:117] "RemoveContainer" containerID="15f1ae3257a2e5f24700977a1d0ca4f15af99c823e5be9305782b7f7bd759ddd" Jul 30 21:27:20.218761 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.218736 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:20.218839 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.218804 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:20.218839 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:20.218835 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:21.137175 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:21.137127 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:21.225872 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:21.225839 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerStarted","Data":"23736bd29fadc20e71c059b77794fd6bb94a2c74e026613698ca7a60c6a29b8c"} Jul 30 21:27:22.035350 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:22.035310 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:22.036569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:22.036541 2567 patch_prober.go:28] interesting pod/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 30 21:27:22.036684 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:22.036620 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 30 21:27:23.034780 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:23.034740 2567 patch_prober.go:28] interesting pod/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 30 21:27:23.035153 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:23.034805 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 30 21:27:24.035131 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:24.035093 2567 patch_prober.go:28] interesting pod/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 30 21:27:24.035534 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:24.035149 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 30 21:27:25.034935 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:25.034895 2567 patch_prober.go:28] interesting pod/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 30 21:27:25.035120 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:25.034966 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 30 21:27:26.035554 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:26.035519 2567 patch_prober.go:28] interesting pod/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 30 21:27:26.035952 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:26.035601 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 30 21:27:27.038894 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:27.038864 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:27.039270 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:27.039124 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:27.039728 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:27.039712 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:36.202958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:36.202921 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:36.203596 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:36.203184 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" containerID="cri-o://23736bd29fadc20e71c059b77794fd6bb94a2c74e026613698ca7a60c6a29b8c" gracePeriod=30 Jul 30 21:27:37.024220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.024181 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:27:37.024743 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.024718 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:27:37.024743 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.024738 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:27:37.024854 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.024816 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:27:37.024854 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.024827 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:27:37.026984 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.026969 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.029137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.029114 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-55614fcd-openshift-default-dockercfg-g229h\"" Jul 30 21:27:37.041351 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.041327 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:27:37.144805 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144770 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.144805 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144809 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144830 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144904 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b8h7\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144956 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.144993 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145168 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.145037 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145168 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.145067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.145168 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.145096 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.245642 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245663 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245693 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245856 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245924 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.245958 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246015 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b8h7\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246105 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246653 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246148 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246653 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246391 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.246653 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.246481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.248057 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.248038 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.248351 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.248331 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.253465 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.253447 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.253660 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.253639 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b8h7\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7\") pod \"e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.340749 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.340669 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:37.476541 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.476515 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:27:37.479361 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:27:37.479333 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8aa10ab_f5e0_4009_8d86_aec4caabf4b9.slice/crio-2a15f85ad26265f76957e5cecfeff8f01dc64c2abfd800b3eba3cdca75e57d23 WatchSource:0}: Error finding container 2a15f85ad26265f76957e5cecfeff8f01dc64c2abfd800b3eba3cdca75e57d23: Status 404 returned error can't find the container with id 2a15f85ad26265f76957e5cecfeff8f01dc64c2abfd800b3eba3cdca75e57d23 Jul 30 21:27:37.481517 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.481488 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:37.481613 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.481545 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:37.481613 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:37.481595 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:38.328168 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:38.328131 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerStarted","Data":"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5"} Jul 30 21:27:38.328535 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:38.328184 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerStarted","Data":"2a15f85ad26265f76957e5cecfeff8f01dc64c2abfd800b3eba3cdca75e57d23"} Jul 30 21:27:38.341771 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:38.341748 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:38.345469 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:38.345241 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podStartSLOduration=1.345222993 podStartE2EDuration="1.345222993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:27:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:27:38.34451381 +0000 UTC m=+1829.043649715" watchObservedRunningTime="2026-07-30 21:27:38.345222993 +0000 UTC m=+1829.044358883" Jul 30 21:27:38.351906 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:38.351876 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:39.333450 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:39.333420 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:39.334381 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:39.334361 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:41.346164 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.346132 2567 generic.go:356] "Generic (PLEG): container finished" podID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerID="23736bd29fadc20e71c059b77794fd6bb94a2c74e026613698ca7a60c6a29b8c" exitCode=0 Jul 30 21:27:41.346664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.346207 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerDied","Data":"23736bd29fadc20e71c059b77794fd6bb94a2c74e026613698ca7a60c6a29b8c"} Jul 30 21:27:41.346664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.346272 2567 scope.go:117] "RemoveContainer" containerID="15f1ae3257a2e5f24700977a1d0ca4f15af99c823e5be9305782b7f7bd759ddd" Jul 30 21:27:41.463454 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.463432 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:41.589128 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589094 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589128 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589134 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589154 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589188 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589213 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589229 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589253 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589340 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4nhp\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589393 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589359 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:41.589769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589403 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:41.589769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589427 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy\") pod \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\" (UID: \"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1\") " Jul 30 21:27:41.589769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589531 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:41.589769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589547 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:41.589988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589794 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:27:41.589988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589803 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.589988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589833 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.589988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589845 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.589988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.589860 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.591616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.591572 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:27:41.591742 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.591644 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp" (OuterVolumeSpecName: "kube-api-access-w4nhp") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "kube-api-access-w4nhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:27:41.591851 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.591828 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:27:41.591851 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.591841 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" (UID: "1d14e8db-de3b-49f1-a5f9-0d34b189b6d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:27:41.690468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.690438 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4nhp\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-kube-api-access-w4nhp\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.690468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.690463 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.690468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.690472 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.690710 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.690480 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:41.690710 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:41.690490 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:27:42.352676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:42.352642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" event={"ID":"1d14e8db-de3b-49f1-a5f9-0d34b189b6d1","Type":"ContainerDied","Data":"3e9f3e9762b61ac0977b4407861cc6fbb2e2750a38038cec7f12058a37fc1637"} Jul 30 21:27:42.352676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:42.352671 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4" Jul 30 21:27:42.353162 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:42.352700 2567 scope.go:117] "RemoveContainer" containerID="23736bd29fadc20e71c059b77794fd6bb94a2c74e026613698ca7a60c6a29b8c" Jul 30 21:27:42.368476 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:42.368452 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:42.371340 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:42.371316 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-60d76360-gw-openshift-default-5db98bd8d-nqbp4"] Jul 30 21:27:43.757871 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:43.757827 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" path="/var/lib/kubelet/pods/1d14e8db-de3b-49f1-a5f9-0d34b189b6d1/volumes" Jul 30 21:27:54.335423 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.335383 2567 patch_prober.go:28] interesting pod/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 30 21:27:54.335893 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.335441 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 30 21:27:54.430001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.429964 2567 generic.go:356] "Generic (PLEG): container finished" podID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerID="2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5" exitCode=0 Jul 30 21:27:54.430155 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.430042 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerDied","Data":"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5"} Jul 30 21:27:54.430442 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.430428 2567 scope.go:117] "RemoveContainer" containerID="2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5" Jul 30 21:27:54.431473 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.431449 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:54.431529 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.431513 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:54.431567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:54.431546 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:27:55.437044 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:55.437009 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerStarted","Data":"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e"} Jul 30 21:27:56.341571 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:56.341530 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:27:56.341886 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:56.341860 2567 patch_prober.go:28] interesting pod/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 30 21:27:56.341947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:56.341922 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 30 21:27:57.341940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:57.341902 2567 patch_prober.go:28] interesting pod/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 30 21:27:57.342412 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:57.341969 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 30 21:27:58.341465 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:58.341423 2567 patch_prober.go:28] interesting pod/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 30 21:27:58.341715 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:58.341501 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 30 21:27:59.341468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:59.341429 2567 patch_prober.go:28] interesting pod/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 30 21:27:59.341896 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:27:59.341515 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 30 21:28:00.346341 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:00.346306 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:28:00.346797 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:00.346703 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:28:00.347456 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:00.347439 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:28:22.832499 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:22.832465 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:28:22.833037 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:22.832689 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" containerID="cri-o://5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e" gracePeriod=30 Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761383 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761927 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761945 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761960 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761968 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="14d13aea-1b1a-41bb-a4b2-e7cb6b24ea1e" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761989 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.761998 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.762104 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.762794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.762116 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d14e8db-de3b-49f1-a5f9-0d34b189b6d1" containerName="istio-proxy" Jul 30 21:28:23.767551 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.767527 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.768469 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.768442 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:28:23.769905 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.769885 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b3059c6e-openshift-default-dockercfg-l5c9q\"" Jul 30 21:28:23.878925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.878891 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.878961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.878995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879073 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879091 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879122 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pkck\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.879296 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.879213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.979960 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.979918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.979961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980059 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980239 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980123 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980239 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980143 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980239 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980165 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pkck\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980239 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980356 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980408 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980467 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980755 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980906 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980854 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.980976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.980912 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.981134 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.981104 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.982437 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.982407 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.982789 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.982767 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.986756 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.986723 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:23.986945 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:23.986925 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pkck\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck\") pod \"e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:24.079816 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.079754 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:24.220250 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.219785 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:28:24.222847 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:28:24.222806 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64bbc7a_092b_4601_87f0_3aa71e3669d6.slice/crio-88e1ecef10ff101e416f79d37acb13e41c160e363095c0c58ca9f2823a1b05c4 WatchSource:0}: Error finding container 88e1ecef10ff101e416f79d37acb13e41c160e363095c0c58ca9f2823a1b05c4: Status 404 returned error can't find the container with id 88e1ecef10ff101e416f79d37acb13e41c160e363095c0c58ca9f2823a1b05c4 Jul 30 21:28:24.225317 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.225285 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:24.225396 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.225351 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:24.225396 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.225380 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:24.600517 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.600478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerStarted","Data":"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2"} Jul 30 21:28:24.600708 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.600558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerStarted","Data":"88e1ecef10ff101e416f79d37acb13e41c160e363095c0c58ca9f2823a1b05c4"} Jul 30 21:28:24.618230 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:24.618178 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podStartSLOduration=1.618163017 podStartE2EDuration="1.618163017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:28:24.616125048 +0000 UTC m=+1875.315260950" watchObservedRunningTime="2026-07-30 21:28:24.618163017 +0000 UTC m=+1875.317298906" Jul 30 21:28:25.080129 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:25.080095 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:25.085166 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:25.085137 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:25.606274 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:25.606239 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:25.607088 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:25.607071 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:28.081251 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.081229 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:28:28.224059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.223970 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224015 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224083 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224123 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224152 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224184 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b8h7\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224228 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224270 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224617 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224318 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket\") pod \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\" (UID: \"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9\") " Jul 30 21:28:28.224617 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224390 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:28:28.224617 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224471 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:28:28.224792 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224654 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:28:28.224792 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224777 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.224902 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224796 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.224902 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224810 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.224989 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224894 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:28:28.224989 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.224836 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:28:28.226357 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.226335 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:28:28.226458 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.226372 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:28:28.226777 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.226759 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:28:28.226828 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.226797 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7" (OuterVolumeSpecName: "kube-api-access-8b8h7") pod "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" (UID: "a8aa10ab-f5e0-4009-8d86-aec4caabf4b9"). InnerVolumeSpecName "kube-api-access-8b8h7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:28:28.326232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326193 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.326232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326223 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.326232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326234 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.326477 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326245 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b8h7\" (UniqueName: \"kubernetes.io/projected/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-kube-api-access-8b8h7\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.326477 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326260 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.326477 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.326271 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:28:28.624972 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.624932 2567 generic.go:356] "Generic (PLEG): container finished" podID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerID="5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e" exitCode=0 Jul 30 21:28:28.625185 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.624984 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" Jul 30 21:28:28.625185 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.625009 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerDied","Data":"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e"} Jul 30 21:28:28.625185 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.625074 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh" event={"ID":"a8aa10ab-f5e0-4009-8d86-aec4caabf4b9","Type":"ContainerDied","Data":"2a15f85ad26265f76957e5cecfeff8f01dc64c2abfd800b3eba3cdca75e57d23"} Jul 30 21:28:28.625185 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.625105 2567 scope.go:117] "RemoveContainer" containerID="5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e" Jul 30 21:28:28.635206 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.635190 2567 scope.go:117] "RemoveContainer" containerID="2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5" Jul 30 21:28:28.643667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.643562 2567 scope.go:117] "RemoveContainer" containerID="5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e" Jul 30 21:28:28.643961 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:28:28.643917 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e\": container with ID starting with 5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e not found: ID does not exist" containerID="5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e" Jul 30 21:28:28.644099 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.643962 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e"} err="failed to get container status \"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e\": rpc error: code = NotFound desc = could not find container \"5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e\": container with ID starting with 5f7250ab8648a10a364b10aaaccbf3bffa4652c6105ecbf0f8ab5dbff53da22e not found: ID does not exist" Jul 30 21:28:28.644099 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.643989 2567 scope.go:117] "RemoveContainer" containerID="2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5" Jul 30 21:28:28.644335 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:28:28.644305 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5\": container with ID starting with 2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5 not found: ID does not exist" containerID="2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5" Jul 30 21:28:28.644414 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.644343 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5"} err="failed to get container status \"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5\": rpc error: code = NotFound desc = could not find container \"2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5\": container with ID starting with 2865619c87b43d8fc28ec3d4891c26f948406c14630c82b82c65f7cfa59c9da5 not found: ID does not exist" Jul 30 21:28:28.646950 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.646929 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:28:28.650710 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:28.650690 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-55614fcd-openshift-default-bd4bd67d4-dbvbh"] Jul 30 21:28:29.758881 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:29.758835 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" path="/var/lib/kubelet/pods/a8aa10ab-f5e0-4009-8d86-aec4caabf4b9/volumes" Jul 30 21:28:39.690559 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.690523 2567 generic.go:356] "Generic (PLEG): container finished" podID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerID="9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2" exitCode=0 Jul 30 21:28:39.691087 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.690621 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerDied","Data":"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2"} Jul 30 21:28:39.691180 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.691163 2567 scope.go:117] "RemoveContainer" containerID="9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2" Jul 30 21:28:39.692372 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.692340 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:39.692467 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.692418 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:39.692467 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:39.692451 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:28:40.606734 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:40.606677 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:40.708034 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:40.707998 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerStarted","Data":"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b"} Jul 30 21:28:41.080497 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:41.080459 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:41.081689 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:41.081663 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 30 21:28:41.081800 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:41.081713 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 30 21:28:42.080883 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:42.080846 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 30 21:28:42.081293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:42.080912 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 30 21:28:43.081019 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:43.080980 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 30 21:28:43.081407 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:43.081036 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 30 21:28:44.080786 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:44.080749 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 30 21:28:44.080974 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:44.080804 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 30 21:28:45.080880 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:45.080841 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 30 21:28:45.081280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:45.080905 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 30 21:28:46.084402 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:46.084372 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:46.084879 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:46.084622 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:46.085444 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:46.085427 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:28:57.896744 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:57.896700 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:28:57.897305 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:28:57.896982 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" containerID="cri-o://2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b" gracePeriod=30 Jul 30 21:29:01.087206 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:01.087168 2567 patch_prober.go:28] interesting pod/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 21:29:01.087640 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:01.087243 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 21:29:03.140025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.140003 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:29:03.263371 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263338 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pkck\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263386 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263417 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263445 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263474 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263634 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263710 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263781 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263788 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263825 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy\") pod \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\" (UID: \"b64bbc7a-092b-4601-87f0-3aa71e3669d6\") " Jul 30 21:29:03.263933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.263825 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264039 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264123 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264180 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264188 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264195 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264230 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.264246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.264247 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.266112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.266086 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:29:03.266112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.266091 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck" (OuterVolumeSpecName: "kube-api-access-7pkck") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "kube-api-access-7pkck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:29:03.266249 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.266147 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:29:03.266249 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.266179 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b64bbc7a-092b-4601-87f0-3aa71e3669d6" (UID: "b64bbc7a-092b-4601-87f0-3aa71e3669d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:29:03.365027 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.364994 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.365027 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.365022 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.365027 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.365032 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.365262 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.365040 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b64bbc7a-092b-4601-87f0-3aa71e3669d6-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.365262 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.365049 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pkck\" (UniqueName: \"kubernetes.io/projected/b64bbc7a-092b-4601-87f0-3aa71e3669d6-kube-api-access-7pkck\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:29:03.839967 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.839933 2567 generic.go:356] "Generic (PLEG): container finished" podID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerID="2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b" exitCode=0 Jul 30 21:29:03.840157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.840000 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" Jul 30 21:29:03.840157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.840012 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerDied","Data":"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b"} Jul 30 21:29:03.840157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.840074 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr" event={"ID":"b64bbc7a-092b-4601-87f0-3aa71e3669d6","Type":"ContainerDied","Data":"88e1ecef10ff101e416f79d37acb13e41c160e363095c0c58ca9f2823a1b05c4"} Jul 30 21:29:03.840157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.840098 2567 scope.go:117] "RemoveContainer" containerID="2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b" Jul 30 21:29:03.849427 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.849411 2567 scope.go:117] "RemoveContainer" containerID="9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2" Jul 30 21:29:03.855985 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.855958 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:29:03.858072 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.858045 2567 scope.go:117] "RemoveContainer" containerID="2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b" Jul 30 21:29:03.858362 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:29:03.858342 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b\": container with ID starting with 2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b not found: ID does not exist" containerID="2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b" Jul 30 21:29:03.858441 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.858373 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b"} err="failed to get container status \"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b\": rpc error: code = NotFound desc = could not find container \"2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b\": container with ID starting with 2093374e083a5c5cbba29a261ca88dee1bbc11ad5aec659820110ffa8e4bdc7b not found: ID does not exist" Jul 30 21:29:03.858441 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.858397 2567 scope.go:117] "RemoveContainer" containerID="9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2" Jul 30 21:29:03.858693 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:29:03.858671 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2\": container with ID starting with 9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2 not found: ID does not exist" containerID="9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2" Jul 30 21:29:03.858840 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.858812 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2"} err="failed to get container status \"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2\": rpc error: code = NotFound desc = could not find container \"9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2\": container with ID starting with 9d678ff11faa9b64695c9eb8d2fa1969fd1fcfffac1c2dc21b7417d3ea417ce2 not found: ID does not exist" Jul 30 21:29:03.860836 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:03.860813 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b3059c6e-openshift-default-744ff698b4-pqnbr"] Jul 30 21:29:05.758476 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:29:05.758439 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" path="/var/lib/kubelet/pods/b64bbc7a-092b-4601-87f0-3aa71e3669d6/volumes" Jul 30 21:30:56.231664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.231630 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:30:56.232229 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232206 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:30:56.232316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232232 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:30:56.232316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232243 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:30:56.232316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232252 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:30:56.232316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232279 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:30:56.232316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232288 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:30:56.232561 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232394 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:30:56.232561 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232408 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:30:56.232561 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.232418 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:30:56.234635 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.234612 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.236900 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.236879 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c76a3a24-openshift-default-dockercfg-66qtz\"" Jul 30 21:30:56.249666 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.249640 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:30:56.355408 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355575 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355414 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355575 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355479 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlwbg\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355575 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355736 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355736 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355693 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355736 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355856 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.355856 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.355774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456446 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456486 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456502 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456544 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456571 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.456651 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456622 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456659 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwbg\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.456925 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457208 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.457101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457208 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.457124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457208 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.457151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.457629 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.457610 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.458692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.458674 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.459004 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.458986 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.465513 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.465492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.465639 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.465514 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlwbg\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg\") pod \"e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.544961 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.544933 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:56.688837 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.688777 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:30:56.690724 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:30:56.690684 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed26c02_db29_4719_bfc1_f11dbeebd428.slice/crio-73f93ed9c88de56e9694f0bcfaddbd7470582447712062595f470ae2a911e379 WatchSource:0}: Error finding container 73f93ed9c88de56e9694f0bcfaddbd7470582447712062595f470ae2a911e379: Status 404 returned error can't find the container with id 73f93ed9c88de56e9694f0bcfaddbd7470582447712062595f470ae2a911e379 Jul 30 21:30:56.692833 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.692796 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:30:56.692933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.692875 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:30:56.692933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:56.692919 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:30:57.459454 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:57.459412 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerStarted","Data":"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9"} Jul 30 21:30:57.459937 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:57.459479 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerStarted","Data":"73f93ed9c88de56e9694f0bcfaddbd7470582447712062595f470ae2a911e379"} Jul 30 21:30:57.477452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:57.477405 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podStartSLOduration=1.477390121 podStartE2EDuration="1.477390121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:30:57.476120643 +0000 UTC m=+2028.175256534" watchObservedRunningTime="2026-07-30 21:30:57.477390121 +0000 UTC m=+2028.176526011" Jul 30 21:30:57.545755 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:57.545726 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:57.550621 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:57.550597 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:58.465195 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:58.465158 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:30:58.466306 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:30:58.466290 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:14.506776 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:14.506734 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 21:31:14.507179 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:14.506824 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 21:31:15.570515 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.570483 2567 generic.go:356] "Generic (PLEG): container finished" podID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerID="2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9" exitCode=0 Jul 30 21:31:15.570958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.570542 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerDied","Data":"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9"} Jul 30 21:31:15.570958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.570895 2567 scope.go:117] "RemoveContainer" containerID="2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9" Jul 30 21:31:15.571947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.571917 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:31:15.572009 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.571995 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:31:15.572063 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:15.572047 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:31:16.577799 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:16.577763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerStarted","Data":"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18"} Jul 30 21:31:17.545285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:17.545246 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:17.545465 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:17.545411 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 30 21:31:17.545516 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:17.545456 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 30 21:31:18.545795 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:18.545754 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 30 21:31:18.546200 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:18.545817 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 30 21:31:19.545852 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:19.545813 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 30 21:31:19.546277 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:19.545874 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 30 21:31:20.545383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:20.545340 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 30 21:31:20.545623 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:20.545400 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 30 21:31:21.545436 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:21.545393 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 30 21:31:21.545873 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:21.545456 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 30 21:31:22.549415 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:22.549379 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:22.549845 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:22.549613 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:22.550193 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:22.550175 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:51.102888 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:51.102801 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:31:51.103437 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:51.103093 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" containerID="cri-o://ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18" gracePeriod=30 Jul 30 21:31:52.551191 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:52.551150 2567 patch_prober.go:28] interesting pod/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 21:31:52.551557 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:52.551222 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 21:31:56.349825 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.349800 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:56.401041 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.400969 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401041 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401028 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401071 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401107 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlwbg\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401149 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401232 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401173 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401232 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401266 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401268 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:31:56.401449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401329 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data\") pod \"9ed26c02-db29-4719-bfc1-f11dbeebd428\" (UID: \"9ed26c02-db29-4719-bfc1-f11dbeebd428\") " Jul 30 21:31:56.401449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401403 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:31:56.401744 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401530 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:31:56.401744 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401641 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.401744 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401661 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed26c02-db29-4719-bfc1-f11dbeebd428-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.401744 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401678 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:31:56.401895 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.401751 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:31:56.403275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.403246 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:31:56.403275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.403256 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:31:56.403419 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.403286 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg" (OuterVolumeSpecName: "kube-api-access-wlwbg") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "kube-api-access-wlwbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:31:56.403537 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.403519 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ed26c02-db29-4719-bfc1-f11dbeebd428" (UID: "9ed26c02-db29-4719-bfc1-f11dbeebd428"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:31:56.502090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502061 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502087 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlwbg\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-kube-api-access-wlwbg\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502098 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502107 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502115 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502124 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.502257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.502131 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed26c02-db29-4719-bfc1-f11dbeebd428-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:31:56.799094 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.799060 2567 generic.go:356] "Generic (PLEG): container finished" podID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerID="ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18" exitCode=0 Jul 30 21:31:56.799260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.799121 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" Jul 30 21:31:56.799260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.799146 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerDied","Data":"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18"} Jul 30 21:31:56.799260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.799197 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27" event={"ID":"9ed26c02-db29-4719-bfc1-f11dbeebd428","Type":"ContainerDied","Data":"73f93ed9c88de56e9694f0bcfaddbd7470582447712062595f470ae2a911e379"} Jul 30 21:31:56.799260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.799219 2567 scope.go:117] "RemoveContainer" containerID="ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18" Jul 30 21:31:56.808709 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.808688 2567 scope.go:117] "RemoveContainer" containerID="2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9" Jul 30 21:31:56.817362 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.817330 2567 scope.go:117] "RemoveContainer" containerID="ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18" Jul 30 21:31:56.817690 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:31:56.817658 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18\": container with ID starting with ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18 not found: ID does not exist" containerID="ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18" Jul 30 21:31:56.817790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.817700 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18"} err="failed to get container status \"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18\": rpc error: code = NotFound desc = could not find container \"ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18\": container with ID starting with ee1e9582456048fff09896f5eb979d01aae7c818f7a7a8b8de6da333daccdd18 not found: ID does not exist" Jul 30 21:31:56.817790 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.817722 2567 scope.go:117] "RemoveContainer" containerID="2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9" Jul 30 21:31:56.818115 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:31:56.818092 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9\": container with ID starting with 2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9 not found: ID does not exist" containerID="2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9" Jul 30 21:31:56.818205 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.818116 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9"} err="failed to get container status \"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9\": rpc error: code = NotFound desc = could not find container \"2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9\": container with ID starting with 2bad262d08dd0b9e8405c160c5f39b68fdd93c7de03bf244c0a8af2a460598d9 not found: ID does not exist" Jul 30 21:31:56.820096 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.820076 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:31:56.823300 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:56.823281 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c76a3a24-openshift-default-7bb96d4696-9sq27"] Jul 30 21:31:57.761692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:31:57.761659 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" path="/var/lib/kubelet/pods/9ed26c02-db29-4719-bfc1-f11dbeebd428/volumes" Jul 30 21:32:09.894340 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:32:09.894308 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:32:09.903901 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:32:09.903877 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:34:09.910117 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.910075 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:34:09.910969 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.910944 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.911080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.910973 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.911080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911008 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.911080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911017 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.911080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911053 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:34:09.911080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911063 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64bbc7a-092b-4601-87f0-3aa71e3669d6" containerName="istio-proxy" Jul 30 21:34:09.911306 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911241 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8aa10ab-f5e0-4009-8d86-aec4caabf4b9" containerName="istio-proxy" Jul 30 21:34:09.911306 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911257 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.911306 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.911276 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed26c02-db29-4719-bfc1-f11dbeebd428" containerName="istio-proxy" Jul 30 21:34:09.917164 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.917134 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:09.920042 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.920015 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f08baaf8-openshift-default-dockercfg-fffkn\"" Jul 30 21:34:09.923399 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:09.923375 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:34:10.050025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.049984 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050222 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050040 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050222 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050089 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050222 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050136 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9wnm\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050222 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050423 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050423 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050313 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050423 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.050527 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.050424 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151313 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151507 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151332 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151507 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151369 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151507 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151507 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151507 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151614 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9wnm\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151982 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151847 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.151982 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.151882 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.152135 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.152112 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.152243 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.152223 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.152502 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.152473 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.154063 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.154043 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.154158 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.154045 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.159925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.159624 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.159925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.159976 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9wnm\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm\") pod \"e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.229836 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.229755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:10.371527 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.371503 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:34:10.372760 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:34:10.372732 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69b1b25_34da_43a5_b920_59deb6c143bd.slice/crio-a94cee2f6ef31a9e7e0ad2a69471cec81eab97406b22fdfe0b980370a68392bf WatchSource:0}: Error finding container a94cee2f6ef31a9e7e0ad2a69471cec81eab97406b22fdfe0b980370a68392bf: Status 404 returned error can't find the container with id a94cee2f6ef31a9e7e0ad2a69471cec81eab97406b22fdfe0b980370a68392bf Jul 30 21:34:10.374948 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.374912 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:10.375039 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.374981 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:10.375039 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.375012 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:10.493836 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.493737 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerStarted","Data":"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524"} Jul 30 21:34:10.493836 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.493798 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerStarted","Data":"a94cee2f6ef31a9e7e0ad2a69471cec81eab97406b22fdfe0b980370a68392bf"} Jul 30 21:34:10.514038 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:10.513984 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podStartSLOduration=1.513966973 podStartE2EDuration="1.513966973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:34:10.512234294 +0000 UTC m=+2221.211370185" watchObservedRunningTime="2026-07-30 21:34:10.513966973 +0000 UTC m=+2221.213102914" Jul 30 21:34:11.230331 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:11.230299 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:11.235395 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:11.235366 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:11.499212 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:11.499187 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:11.500233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:11.500213 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:27.540725 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:27.540688 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 30 21:34:27.541285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:27.540745 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 30 21:34:28.591707 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.591623 2567 generic.go:356] "Generic (PLEG): container finished" podID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerID="90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524" exitCode=0 Jul 30 21:34:28.591707 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.591690 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerDied","Data":"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524"} Jul 30 21:34:28.592200 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.592064 2567 scope.go:117] "RemoveContainer" containerID="90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524" Jul 30 21:34:28.593143 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.593107 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:28.593226 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.593205 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:28.593273 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:28.593240 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:34:29.602440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:29.602395 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerStarted","Data":"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477"} Jul 30 21:34:30.230594 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:30.230553 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:30.230814 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:30.230793 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 30 21:34:30.230880 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:30.230856 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 30 21:34:31.230951 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:31.230917 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 30 21:34:31.231318 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:31.230982 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 30 21:34:32.230221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:32.230137 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 30 21:34:32.230221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:32.230205 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 30 21:34:33.230243 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:33.230200 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 30 21:34:33.230692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:33.230262 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 30 21:34:34.231025 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:34.230985 2567 patch_prober.go:28] interesting pod/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 30 21:34:34.231431 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:34.231039 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 30 21:34:35.234355 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:35.234325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:35.234760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:35.234605 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:34:35.235287 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:34:35.235269 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:36:24.370407 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:24.370310 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:36:24.371029 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:24.370562 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" containerID="cri-o://eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477" gracePeriod=30 Jul 30 21:36:29.620271 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.620246 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:36:29.722011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.721923 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.721965 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.721996 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722035 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722105 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722169 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722202 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722218 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722243 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9wnm\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722277 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy\") pod \"a69b1b25-34da-43a5-b920-59deb6c143bd\" (UID: \"a69b1b25-34da-43a5-b920-59deb6c143bd\") " Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722374 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722393 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722507 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722631 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722660 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722686 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b1b25-34da-43a5-b920-59deb6c143bd-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722703 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:36:29.722801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.722714 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.724347 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.724319 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:36:29.724457 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.724431 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:36:29.724503 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.724431 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:36:29.724503 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.724457 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm" (OuterVolumeSpecName: "kube-api-access-g9wnm") pod "a69b1b25-34da-43a5-b920-59deb6c143bd" (UID: "a69b1b25-34da-43a5-b920-59deb6c143bd"). InnerVolumeSpecName "kube-api-access-g9wnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:36:29.823849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.823816 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.823849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.823842 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.823849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.823854 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.824072 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.823864 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9wnm\" (UniqueName: \"kubernetes.io/projected/a69b1b25-34da-43a5-b920-59deb6c143bd-kube-api-access-g9wnm\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:29.824072 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:29.823873 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b1b25-34da-43a5-b920-59deb6c143bd-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:36:30.272177 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.272141 2567 generic.go:356] "Generic (PLEG): container finished" podID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerID="eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477" exitCode=0 Jul 30 21:36:30.272342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.272190 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" Jul 30 21:36:30.272342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.272228 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerDied","Data":"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477"} Jul 30 21:36:30.272342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.272291 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk" event={"ID":"a69b1b25-34da-43a5-b920-59deb6c143bd","Type":"ContainerDied","Data":"a94cee2f6ef31a9e7e0ad2a69471cec81eab97406b22fdfe0b980370a68392bf"} Jul 30 21:36:30.272342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.272323 2567 scope.go:117] "RemoveContainer" containerID="eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477" Jul 30 21:36:30.281649 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.281634 2567 scope.go:117] "RemoveContainer" containerID="90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524" Jul 30 21:36:30.289892 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.289866 2567 scope.go:117] "RemoveContainer" containerID="eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477" Jul 30 21:36:30.290217 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.290192 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:36:30.290320 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:36:30.290189 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477\": container with ID starting with eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477 not found: ID does not exist" containerID="eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477" Jul 30 21:36:30.290320 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.290254 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477"} err="failed to get container status \"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477\": rpc error: code = NotFound desc = could not find container \"eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477\": container with ID starting with eeef277c1c05351d0151c28a1e7b1b400cebf8e45f8c6bee7d5e77f58395e477 not found: ID does not exist" Jul 30 21:36:30.290320 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.290276 2567 scope.go:117] "RemoveContainer" containerID="90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524" Jul 30 21:36:30.290629 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:36:30.290602 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524\": container with ID starting with 90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524 not found: ID does not exist" containerID="90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524" Jul 30 21:36:30.290732 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.290654 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524"} err="failed to get container status \"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524\": rpc error: code = NotFound desc = could not find container \"90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524\": container with ID starting with 90d7201e7a72d92a4671b5139d39073b05e6409d53ea81d7de26daebf259e524 not found: ID does not exist" Jul 30 21:36:30.293067 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:30.293046 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f08baaf8-openshift-default-dc84989d8-dxthk"] Jul 30 21:36:31.758865 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:36:31.758825 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" path="/var/lib/kubelet/pods/a69b1b25-34da-43a5-b920-59deb6c143bd/volumes" Jul 30 21:37:09.929298 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:09.929271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:37:09.939644 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:09.939622 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:37:29.619008 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.618976 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:37:29.619466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.619329 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:29.619466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.619340 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:29.619466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.619362 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:29.619466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.619367 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:29.619466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.619429 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:29.622459 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.622443 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.624689 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.624667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b2dab5-openshift-default-dockercfg-xc8rr\"" Jul 30 21:37:29.636098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.636072 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:37:29.686155 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686314 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686168 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686314 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686314 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686283 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686487 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686487 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686385 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686487 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686418 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzpx2\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686487 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686480 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.686733 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.686510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.787964 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.787931 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.787964 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.787967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.787988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzpx2\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788165 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788203 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788197 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788575 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788510 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788613 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788736 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788677 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.788736 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.788724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.789099 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.789077 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.790302 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.790279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.790730 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.790703 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.798796 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.798776 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzpx2\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.798897 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.798781 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token\") pod \"e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:29.934079 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:29.933996 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:30.074046 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.074015 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:37:30.075428 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:37:30.075399 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442801f5_60af_470d_a82b_8831d6cd35fb.slice/crio-3163b85389278a90c915ce003ddb449eb0f9ad92e292e12d87039c061a2e1b1b WatchSource:0}: Error finding container 3163b85389278a90c915ce003ddb449eb0f9ad92e292e12d87039c061a2e1b1b: Status 404 returned error can't find the container with id 3163b85389278a90c915ce003ddb449eb0f9ad92e292e12d87039c061a2e1b1b Jul 30 21:37:30.077341 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.077308 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:30.077434 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.077373 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:30.077434 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.077403 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:30.598194 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.598158 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerStarted","Data":"37ff721782193ec9cc4358f6ab5c12e5fd858b8a187055b2ac02ebfa1c43f1e9"} Jul 30 21:37:30.598398 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.598216 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerStarted","Data":"3163b85389278a90c915ce003ddb449eb0f9ad92e292e12d87039c061a2e1b1b"} Jul 30 21:37:30.618211 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.618163 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podStartSLOduration=1.6181497299999998 podStartE2EDuration="1.61814973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:37:30.614596057 +0000 UTC m=+2421.313731941" watchObservedRunningTime="2026-07-30 21:37:30.61814973 +0000 UTC m=+2421.317285621" Jul 30 21:37:30.934168 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:30.934084 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:31.940194 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:31.940118 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:32.609337 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:32.609306 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:32.610228 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:32.610212 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:46.695281 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.695242 2567 generic.go:356] "Generic (PLEG): container finished" podID="442801f5-60af-470d-a82b-8831d6cd35fb" containerID="37ff721782193ec9cc4358f6ab5c12e5fd858b8a187055b2ac02ebfa1c43f1e9" exitCode=0 Jul 30 21:37:46.695777 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.695314 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerDied","Data":"37ff721782193ec9cc4358f6ab5c12e5fd858b8a187055b2ac02ebfa1c43f1e9"} Jul 30 21:37:46.695777 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.695718 2567 scope.go:117] "RemoveContainer" containerID="37ff721782193ec9cc4358f6ab5c12e5fd858b8a187055b2ac02ebfa1c43f1e9" Jul 30 21:37:46.696882 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.696850 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:46.696951 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.696932 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:46.696990 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:46.696965 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:47.609963 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:47.609892 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:47.706774 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:47.706734 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerStarted","Data":"84a895f3faded017b64245842f51aa42ca314207b80a6f2bd7d54fa003bfa42f"} Jul 30 21:37:47.934793 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:47.934698 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:47.935883 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:47.935858 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 30 21:37:47.936005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:47.935914 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 30 21:37:48.934929 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:48.934895 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 30 21:37:48.935329 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:48.934949 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 30 21:37:49.935089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:49.935056 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 30 21:37:49.935461 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:49.935110 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 30 21:37:50.935075 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:50.935036 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 30 21:37:50.935257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:50.935112 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 30 21:37:51.934397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:51.934361 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 30 21:37:51.934574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:51.934409 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 30 21:37:52.938282 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:52.938246 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:52.938775 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:52.938614 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:52.939336 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:52.939317 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:37:58.161493 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.161456 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:37:58.162177 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.162155 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69b1b25-34da-43a5-b920-59deb6c143bd" containerName="istio-proxy" Jul 30 21:37:58.165121 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.165101 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.167338 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.167311 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e79f82-openshift-default-dockercfg-cvxpc\"" Jul 30 21:37:58.177695 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.177661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275208 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275259 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275287 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275416 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkdq7\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275563 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.275748 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.275626 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376751 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376760 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkdq7\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376811 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376858 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376875 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.376933 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.376890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377302 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377359 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377359 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377330 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377609 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377564 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377750 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377724 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.377818 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.377733 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.379250 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.379223 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.379425 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.379405 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.384180 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.384153 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.384384 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.384366 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkdq7\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7\") pod \"e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.478622 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.478529 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:58.621730 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.621706 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:37:58.623967 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:37:58.623930 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9734f25_128c_4602_add6_b47366fd2d70.slice/crio-92817d20d0087cff0dde8f3a2c98380ce54d84041918ce84b1457420098a578d WatchSource:0}: Error finding container 92817d20d0087cff0dde8f3a2c98380ce54d84041918ce84b1457420098a578d: Status 404 returned error can't find the container with id 92817d20d0087cff0dde8f3a2c98380ce54d84041918ce84b1457420098a578d Jul 30 21:37:58.626454 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.626417 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:58.626546 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.626485 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:58.626546 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.626518 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:37:58.778018 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.777982 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerStarted","Data":"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7"} Jul 30 21:37:58.778236 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.778041 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerStarted","Data":"92817d20d0087cff0dde8f3a2c98380ce54d84041918ce84b1457420098a578d"} Jul 30 21:37:58.802814 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:58.802757 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podStartSLOduration=0.802736802 podStartE2EDuration="802.736802ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:37:58.797880357 +0000 UTC m=+2449.497016248" watchObservedRunningTime="2026-07-30 21:37:58.802736802 +0000 UTC m=+2449.501872692" Jul 30 21:37:59.479223 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:59.479191 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:59.484091 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:59.484064 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:59.783133 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:59.783107 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:37:59.784111 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:37:59.784089 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:38:14.785401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.785364 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:14.785958 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.785421 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:14.867334 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.867306 2567 generic.go:356] "Generic (PLEG): container finished" podID="e9734f25-128c-4602-add6-b47366fd2d70" containerID="30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7" exitCode=0 Jul 30 21:38:14.867474 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.867372 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerDied","Data":"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7"} Jul 30 21:38:14.867727 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.867713 2567 scope.go:117] "RemoveContainer" containerID="30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7" Jul 30 21:38:14.868721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.868692 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:38:14.868849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.868768 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:38:14.868849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:14.868817 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:38:15.873989 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:15.873956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerStarted","Data":"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19"} Jul 30 21:38:16.478916 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:16.478881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:38:16.479120 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:16.479068 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:16.479253 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:16.479120 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:17.479099 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:17.479062 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:17.479564 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:17.479123 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:18.479401 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:18.479357 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:18.479922 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:18.479426 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:19.479649 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:19.479611 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:19.480042 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:19.479667 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:20.479048 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:20.479014 2567 patch_prober.go:28] interesting pod/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 30 21:38:20.479230 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:20.479070 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 30 21:38:21.483152 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:21.483121 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:38:21.483637 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:21.483395 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:38:21.484137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:38:21.484121 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:40:02.145356 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.145320 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f"] Jul 30 21:40:02.149010 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.148992 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:02.150904 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.150883 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pwwf4\"" Jul 30 21:40:02.155628 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.155601 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f"] Jul 30 21:40:02.261275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.261236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f\" (UID: \"20929627-3670-4f28-8f93-bd753be8a5e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:02.361833 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.361793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f\" (UID: \"20929627-3670-4f28-8f93-bd753be8a5e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:02.362495 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.362467 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f\" (UID: \"20929627-3670-4f28-8f93-bd753be8a5e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:02.461874 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.461792 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:02.598993 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.598965 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f"] Jul 30 21:40:02.600736 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:40:02.600710 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20929627_3670_4f28_8f93_bd753be8a5e6.slice/crio-9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da WatchSource:0}: Error finding container 9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da: Status 404 returned error can't find the container with id 9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da Jul 30 21:40:02.602782 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:02.602766 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 21:40:03.463499 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:03.463461 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" event={"ID":"20929627-3670-4f28-8f93-bd753be8a5e6","Type":"ContainerStarted","Data":"9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da"} Jul 30 21:40:06.491212 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:06.491181 2567 generic.go:356] "Generic (PLEG): container finished" podID="20929627-3670-4f28-8f93-bd753be8a5e6" containerID="55b964410040cf88f5d286ff105afc3967e71daf54339dda5e0f8548f78b1f4d" exitCode=0 Jul 30 21:40:06.491614 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:06.491273 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" event={"ID":"20929627-3670-4f28-8f93-bd753be8a5e6","Type":"ContainerDied","Data":"55b964410040cf88f5d286ff105afc3967e71daf54339dda5e0f8548f78b1f4d"} Jul 30 21:40:07.629773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:07.629752 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:07.713861 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:07.713833 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca\") pod \"20929627-3670-4f28-8f93-bd753be8a5e6\" (UID: \"20929627-3670-4f28-8f93-bd753be8a5e6\") " Jul 30 21:40:07.714171 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:07.714148 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20929627-3670-4f28-8f93-bd753be8a5e6" (UID: "20929627-3670-4f28-8f93-bd753be8a5e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:40:07.815565 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:07.815541 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20929627-3670-4f28-8f93-bd753be8a5e6-maas-api-service-ca\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:08.503475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:08.503443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" event={"ID":"20929627-3670-4f28-8f93-bd753be8a5e6","Type":"ContainerDied","Data":"9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da"} Jul 30 21:40:08.503475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:08.503471 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f" Jul 30 21:40:08.503718 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:08.503496 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b6685e35d205c15f569ce234d369fcd7c150ddea9673de88be8e4f1cc77e4da" Jul 30 21:40:12.868809 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:12.868768 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f"] Jul 30 21:40:12.871493 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:12.871468 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b2dab5-ff66f"] Jul 30 21:40:13.758716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:13.758681 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20929627-3670-4f28-8f93-bd753be8a5e6" path="/var/lib/kubelet/pods/20929627-3670-4f28-8f93-bd753be8a5e6/volumes" Jul 30 21:40:19.424920 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:19.424885 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:40:19.425355 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:19.425123 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" containerID="cri-o://84a895f3faded017b64245842f51aa42ca314207b80a6f2bd7d54fa003bfa42f" gracePeriod=30 Jul 30 21:40:22.940991 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:22.940951 2567 patch_prober.go:28] interesting pod/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 21:40:22.941376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:22.941020 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 21:40:24.591441 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.591404 2567 generic.go:356] "Generic (PLEG): container finished" podID="442801f5-60af-470d-a82b-8831d6cd35fb" containerID="84a895f3faded017b64245842f51aa42ca314207b80a6f2bd7d54fa003bfa42f" exitCode=0 Jul 30 21:40:24.591854 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.591476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerDied","Data":"84a895f3faded017b64245842f51aa42ca314207b80a6f2bd7d54fa003bfa42f"} Jul 30 21:40:24.591854 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.591545 2567 scope.go:117] "RemoveContainer" containerID="37ff721782193ec9cc4358f6ab5c12e5fd858b8a187055b2ac02ebfa1c43f1e9" Jul 30 21:40:24.680968 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.680943 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:40:24.771674 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771646 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.771840 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771686 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzpx2\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.771840 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771750 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.771840 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771790 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.771840 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771835 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.772040 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771857 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.772040 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771897 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.772040 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771930 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.772040 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771998 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs\") pod \"442801f5-60af-470d-a82b-8831d6cd35fb\" (UID: \"442801f5-60af-470d-a82b-8831d6cd35fb\") " Jul 30 21:40:24.772244 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.771895 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:24.772453 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.772372 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:24.772453 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.772382 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:24.772453 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.772408 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.772737 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.772570 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:24.772798 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.772756 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:40:24.774038 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.774015 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:24.774475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.774419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:40:24.774475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.774438 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:40:24.774637 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.774491 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2" (OuterVolumeSpecName: "kube-api-access-hzpx2") pod "442801f5-60af-470d-a82b-8831d6cd35fb" (UID: "442801f5-60af-470d-a82b-8831d6cd35fb"). InnerVolumeSpecName "kube-api-access-hzpx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:40:24.873087 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873058 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzpx2\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-kube-api-access-hzpx2\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873087 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873080 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/442801f5-60af-470d-a82b-8831d6cd35fb-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873087 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873089 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/442801f5-60af-470d-a82b-8831d6cd35fb-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873303 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873098 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873303 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873107 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873303 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873115 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/442801f5-60af-470d-a82b-8831d6cd35fb-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873303 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873122 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:24.873303 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:24.873130 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/442801f5-60af-470d-a82b-8831d6cd35fb-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:25.598628 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.598574 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" event={"ID":"442801f5-60af-470d-a82b-8831d6cd35fb","Type":"ContainerDied","Data":"3163b85389278a90c915ce003ddb449eb0f9ad92e292e12d87039c061a2e1b1b"} Jul 30 21:40:25.598628 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.598613 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c" Jul 30 21:40:25.599133 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.598662 2567 scope.go:117] "RemoveContainer" containerID="84a895f3faded017b64245842f51aa42ca314207b80a6f2bd7d54fa003bfa42f" Jul 30 21:40:25.621985 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.621957 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:40:25.624973 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.624951 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b2dab5-openshift-default-7f84b66bf8-bsj5c"] Jul 30 21:40:25.759982 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:25.759949 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" path="/var/lib/kubelet/pods/442801f5-60af-470d-a82b-8831d6cd35fb/volumes" Jul 30 21:40:36.871680 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:36.871601 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:40:36.872212 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:36.871854 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" containerID="cri-o://d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19" gracePeriod=30 Jul 30 21:40:37.735829 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.735796 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:40:37.736270 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736250 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20929627-3670-4f28-8f93-bd753be8a5e6" containerName="revoke-keys" Jul 30 21:40:37.736432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736274 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="20929627-3670-4f28-8f93-bd753be8a5e6" containerName="revoke-keys" Jul 30 21:40:37.736432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736287 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.736432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736294 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.736432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736305 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.736432 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736313 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.736742 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736437 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.736742 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736448 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="20929627-3670-4f28-8f93-bd753be8a5e6" containerName="revoke-keys" Jul 30 21:40:37.736742 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.736462 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="442801f5-60af-470d-a82b-8831d6cd35fb" containerName="istio-proxy" Jul 30 21:40:37.741001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.740981 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.743006 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.742986 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-db28eb-openshift-default-dockercfg-6ttx6\"" Jul 30 21:40:37.759337 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.759310 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:40:37.775845 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.775819 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.775959 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.775898 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.775959 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.775950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776074 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.775983 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlgg7\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776130 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.776090 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.776140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.776174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.776220 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.776297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.776255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877428 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877447 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlgg7\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877570 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.877930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.878376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.877932 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.878376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.878225 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.878376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.878264 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.878376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.878285 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.878376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.878293 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.880166 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.880136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.880273 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.880184 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.884505 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.884482 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:37.884755 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:37.884739 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlgg7\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7\") pod \"e2e-api-a-db28eb-openshift-default-c577b5f4-dck44\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:38.050988 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.050963 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:38.188919 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.188895 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:40:38.190554 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:40:38.190518 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd96194_3806_42d5_86aa_8573ccbe11d8.slice/crio-6845b1012d79752a412843dab6b4024f75547e831566879a6a6ba6807e45ad59 WatchSource:0}: Error finding container 6845b1012d79752a412843dab6b4024f75547e831566879a6a6ba6807e45ad59: Status 404 returned error can't find the container with id 6845b1012d79752a412843dab6b4024f75547e831566879a6a6ba6807e45ad59 Jul 30 21:40:38.192615 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.192567 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:38.192722 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.192646 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:38.192722 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.192674 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:38.673298 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.673256 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerStarted","Data":"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677"} Jul 30 21:40:38.673455 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.673324 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerStarted","Data":"6845b1012d79752a412843dab6b4024f75547e831566879a6a6ba6807e45ad59"} Jul 30 21:40:38.696616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:38.696554 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podStartSLOduration=1.696538992 podStartE2EDuration="1.696538992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:40:38.692965879 +0000 UTC m=+2609.392101770" watchObservedRunningTime="2026-07-30 21:40:38.696538992 +0000 UTC m=+2609.395674886" Jul 30 21:40:39.051463 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:39.051434 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:39.056368 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:39.056341 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:39.678853 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:39.678822 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:39.679820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:39.679797 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:42.127144 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.127117 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:40:42.220843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220764 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.220843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220797 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.220843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220823 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkdq7\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220853 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220889 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220919 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.220992 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221016 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221049 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs\") pod \"e9734f25-128c-4602-add6-b47366fd2d70\" (UID: \"e9734f25-128c-4602-add6-b47366fd2d70\") " Jul 30 21:40:42.221124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221049 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:42.221604 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221395 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.221604 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221396 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:42.221604 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221430 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:40:42.221604 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221459 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:42.221837 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.221606 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:42.223157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.223131 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:40:42.223157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.223151 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:40:42.223431 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.223404 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:40:42.223431 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.223409 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7" (OuterVolumeSpecName: "kube-api-access-tkdq7") pod "e9734f25-128c-4602-add6-b47366fd2d70" (UID: "e9734f25-128c-4602-add6-b47366fd2d70"). InnerVolumeSpecName "kube-api-access-tkdq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:40:42.322830 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322797 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.322830 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322825 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.322830 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322834 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9734f25-128c-4602-add6-b47366fd2d70-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.323055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322845 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.323055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322853 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9734f25-128c-4602-add6-b47366fd2d70-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.323055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322861 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9734f25-128c-4602-add6-b47366fd2d70-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.323055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322871 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkdq7\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-kube-api-access-tkdq7\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.323055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.322879 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9734f25-128c-4602-add6-b47366fd2d70-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:40:42.696632 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.696595 2567 generic.go:356] "Generic (PLEG): container finished" podID="e9734f25-128c-4602-add6-b47366fd2d70" containerID="d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19" exitCode=0 Jul 30 21:40:42.696776 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.696647 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" Jul 30 21:40:42.696776 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.696654 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerDied","Data":"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19"} Jul 30 21:40:42.696776 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.696710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj" event={"ID":"e9734f25-128c-4602-add6-b47366fd2d70","Type":"ContainerDied","Data":"92817d20d0087cff0dde8f3a2c98380ce54d84041918ce84b1457420098a578d"} Jul 30 21:40:42.696776 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.696741 2567 scope.go:117] "RemoveContainer" containerID="d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19" Jul 30 21:40:42.706224 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.706206 2567 scope.go:117] "RemoveContainer" containerID="30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7" Jul 30 21:40:42.714076 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.714058 2567 scope.go:117] "RemoveContainer" containerID="d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19" Jul 30 21:40:42.714353 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:40:42.714331 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19\": container with ID starting with d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19 not found: ID does not exist" containerID="d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19" Jul 30 21:40:42.714429 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.714361 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19"} err="failed to get container status \"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19\": rpc error: code = NotFound desc = could not find container \"d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19\": container with ID starting with d3be86b0cfa73a79be10f528168b113eee5d4c50579470aabf5762c4239b0d19 not found: ID does not exist" Jul 30 21:40:42.714429 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.714378 2567 scope.go:117] "RemoveContainer" containerID="30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7" Jul 30 21:40:42.714647 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:40:42.714623 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7\": container with ID starting with 30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7 not found: ID does not exist" containerID="30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7" Jul 30 21:40:42.714745 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.714655 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7"} err="failed to get container status \"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7\": rpc error: code = NotFound desc = could not find container \"30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7\": container with ID starting with 30bb0d79e819af9898dca6a7cf02727a1e80e5833bc3e5497257d478e90fc3a7 not found: ID does not exist" Jul 30 21:40:42.717353 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.717334 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:40:42.722962 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:42.722937 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e79f82-openshift-default-d4b67c778-lcdrj"] Jul 30 21:40:43.757495 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:43.757465 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9734f25-128c-4602-add6-b47366fd2d70" path="/var/lib/kubelet/pods/e9734f25-128c-4602-add6-b47366fd2d70/volumes" Jul 30 21:40:55.507800 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.507770 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:47920->10.133.0.54:15021: read: connection reset by peer" start-of-body= Jul 30 21:40:55.508204 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.507848 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:47920->10.133.0.54:15021: read: connection reset by peer" Jul 30 21:40:55.771728 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.771650 2567 generic.go:356] "Generic (PLEG): container finished" podID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerID="0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677" exitCode=0 Jul 30 21:40:55.771873 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.771722 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerDied","Data":"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677"} Jul 30 21:40:55.772108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.772093 2567 scope.go:117] "RemoveContainer" containerID="0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677" Jul 30 21:40:55.773059 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.773024 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:55.773153 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.773093 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:55.773153 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:55.773125 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:40:56.785443 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:56.785407 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerStarted","Data":"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138"} Jul 30 21:40:57.051403 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:57.051306 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:40:57.051667 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:57.051643 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 30 21:40:57.051725 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:57.051706 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 30 21:40:58.051912 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:58.051881 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 30 21:40:58.052292 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:58.051947 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 30 21:40:59.051373 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:59.051331 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 30 21:40:59.051552 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:40:59.051403 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 30 21:41:00.051737 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:00.051698 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 30 21:41:00.052142 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:00.051754 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 30 21:41:01.051330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:01.051291 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 30 21:41:01.051517 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:01.051348 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 30 21:41:02.055007 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:02.054967 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:41:02.055385 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:02.055307 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:41:02.055880 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:02.055861 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:41:06.340311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340272 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340722 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340737 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340752 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340759 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340816 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.340835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.340824 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9734f25-128c-4602-add6-b47366fd2d70" containerName="istio-proxy" Jul 30 21:41:06.342864 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.342846 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.344800 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.344778 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d8e5fa-openshift-default-dockercfg-js2mk\"" Jul 30 21:41:06.354545 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.354519 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:41:06.445221 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445189 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445228 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlt7j\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445433 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.445574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.445496 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546372 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546322 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546489 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlt7j\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546881 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546610 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546881 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546635 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.546881 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546670 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.547041 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.546973 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.547416 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.547255 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.547416 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.547338 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.547644 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.547512 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.547644 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.547553 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.549921 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.549898 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.550112 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.550039 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.554300 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.554280 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.554397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.554370 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlt7j\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j\") pod \"e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.655311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.655239 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:06.790789 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.790763 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:41:06.791465 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:41:06.791440 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6cb5e89_cc45_4bfb_8791_59b99fd77dff.slice/crio-0b739fe73d5a129c76224238f22ffd80f90b1e78e496c52215d479ec893a98eb WatchSource:0}: Error finding container 0b739fe73d5a129c76224238f22ffd80f90b1e78e496c52215d479ec893a98eb: Status 404 returned error can't find the container with id 0b739fe73d5a129c76224238f22ffd80f90b1e78e496c52215d479ec893a98eb Jul 30 21:41:06.793493 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.793459 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:06.793631 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.793535 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:06.793631 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.793595 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:06.851700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:06.851667 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerStarted","Data":"0b739fe73d5a129c76224238f22ffd80f90b1e78e496c52215d479ec893a98eb"} Jul 30 21:41:07.859034 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:07.858998 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerStarted","Data":"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419"} Jul 30 21:41:07.878837 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:07.878788 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podStartSLOduration=1.8787720810000001 podStartE2EDuration="1.878772081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:41:07.876428064 +0000 UTC m=+2638.575563955" watchObservedRunningTime="2026-07-30 21:41:07.878772081 +0000 UTC m=+2638.577908015" Jul 30 21:41:08.655925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:08.655881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:08.661102 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:08.661077 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:08.865182 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:08.865142 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:08.866179 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:08.866161 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:24.532155 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.532120 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:60458->10.133.0.55:15021: read: connection reset by peer" start-of-body= Jul 30 21:41:24.532652 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.532189 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:60458->10.133.0.55:15021: read: connection reset by peer" Jul 30 21:41:24.961345 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.961256 2567 generic.go:356] "Generic (PLEG): container finished" podID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerID="d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419" exitCode=0 Jul 30 21:41:24.961345 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.961329 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerDied","Data":"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419"} Jul 30 21:41:24.961755 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.961741 2567 scope.go:117] "RemoveContainer" containerID="d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419" Jul 30 21:41:24.962768 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.962740 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:24.962846 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.962806 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:24.962846 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:24.962838 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:41:25.969334 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:25.969296 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerStarted","Data":"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b"} Jul 30 21:41:26.656310 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:26.656270 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:26.656568 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:26.656544 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 30 21:41:26.656703 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:26.656625 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 30 21:41:27.656273 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:27.656234 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 30 21:41:27.656711 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:27.656293 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 30 21:41:28.655906 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:28.655870 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 30 21:41:28.656088 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:28.655922 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 30 21:41:29.656485 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:29.656447 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 30 21:41:29.656920 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:29.656504 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 30 21:41:30.655778 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:30.655740 2567 patch_prober.go:28] interesting pod/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 30 21:41:30.655961 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:30.655798 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 30 21:41:31.659631 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:31.659594 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:31.660091 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:31.659875 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:31.660626 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:31.660603 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:41:59.768895 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:59.768860 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:41:59.769479 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:41:59.769094 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" containerID="cri-o://198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138" gracePeriod=30 Jul 30 21:42:02.058257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:02.058168 2567 patch_prober.go:28] interesting pod/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 21:42:02.058257 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:02.058226 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 21:42:05.018293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.018266 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:42:05.105739 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105657 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.105739 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105699 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.105739 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105720 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105772 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105823 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105867 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlgg7\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105905 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105932 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105974 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:05.106015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.105986 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs\") pod \"bbd96194-3806-42d5-86aa-8573ccbe11d8\" (UID: \"bbd96194-3806-42d5-86aa-8573ccbe11d8\") " Jul 30 21:42:05.106377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106088 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:05.106377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106123 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:05.106377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106147 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:05.106377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106360 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.106625 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106380 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.106625 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106383 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:42:05.106625 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106394 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.106625 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.106408 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.107981 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.107949 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:42:05.107981 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.107965 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7" (OuterVolumeSpecName: "kube-api-access-zlgg7") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "kube-api-access-zlgg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:42:05.108290 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.108266 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:42:05.108361 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.108280 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbd96194-3806-42d5-86aa-8573ccbe11d8" (UID: "bbd96194-3806-42d5-86aa-8573ccbe11d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:05.197747 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.197690 2567 generic.go:356] "Generic (PLEG): container finished" podID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerID="198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138" exitCode=0 Jul 30 21:42:05.197941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.197786 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" Jul 30 21:42:05.197941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.197808 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerDied","Data":"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138"} Jul 30 21:42:05.197941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.197867 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44" event={"ID":"bbd96194-3806-42d5-86aa-8573ccbe11d8","Type":"ContainerDied","Data":"6845b1012d79752a412843dab6b4024f75547e831566879a6a6ba6807e45ad59"} Jul 30 21:42:05.197941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.197889 2567 scope.go:117] "RemoveContainer" containerID="198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138" Jul 30 21:42:05.207370 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207354 2567 scope.go:117] "RemoveContainer" containerID="0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677" Jul 30 21:42:05.207434 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207414 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.207434 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207431 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.207530 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207441 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlgg7\" (UniqueName: \"kubernetes.io/projected/bbd96194-3806-42d5-86aa-8573ccbe11d8-kube-api-access-zlgg7\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.207530 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207449 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbd96194-3806-42d5-86aa-8573ccbe11d8-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.207530 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.207457 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbd96194-3806-42d5-86aa-8573ccbe11d8-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:05.215383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.215366 2567 scope.go:117] "RemoveContainer" containerID="198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138" Jul 30 21:42:05.215685 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:42:05.215656 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138\": container with ID starting with 198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138 not found: ID does not exist" containerID="198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138" Jul 30 21:42:05.215843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.215696 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138"} err="failed to get container status \"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138\": rpc error: code = NotFound desc = could not find container \"198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138\": container with ID starting with 198ba617a96cf7578d70e5a67ce4d37c824f549ed5433561861e82b3d0b3a138 not found: ID does not exist" Jul 30 21:42:05.215843 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.215719 2567 scope.go:117] "RemoveContainer" containerID="0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677" Jul 30 21:42:05.216278 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:42:05.216131 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677\": container with ID starting with 0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677 not found: ID does not exist" containerID="0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677" Jul 30 21:42:05.216278 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.216163 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677"} err="failed to get container status \"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677\": rpc error: code = NotFound desc = could not find container \"0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677\": container with ID starting with 0ccfeeb4cf6b2cc3e3f66a0b07f8440592d91d625f57b6b1b66c67d3b5c84677 not found: ID does not exist" Jul 30 21:42:05.218947 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.218927 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:42:05.223005 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.222984 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-db28eb-openshift-default-c577b5f4-dck44"] Jul 30 21:42:05.758316 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:05.758276 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" path="/var/lib/kubelet/pods/bbd96194-3806-42d5-86aa-8573ccbe11d8/volumes" Jul 30 21:42:09.964201 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:09.964174 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:42:09.975124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:09.975102 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:42:22.211520 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:22.211486 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:42:22.212010 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:22.211748 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" containerID="cri-o://dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b" gracePeriod=30 Jul 30 21:42:23.096853 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.096816 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:42:23.097440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.097413 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:23.097440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.097441 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:23.097662 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.097548 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:23.097662 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.097562 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:23.099795 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.099777 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.101777 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.101757 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2abe52-openshift-default-dockercfg-x7tdj\"" Jul 30 21:42:23.109920 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.109895 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:42:23.169661 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169807 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169664 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169807 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169695 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169807 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdggt\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169807 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169784 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169987 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169820 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169987 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169872 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169987 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169947 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.169987 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.169975 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271196 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdggt\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271238 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271389 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271449 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.271763 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271713 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.272175 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271765 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.272175 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.271874 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.272175 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.272064 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.272175 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.272102 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.273781 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.273760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.273911 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.273893 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.278838 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.278818 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.279055 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.279039 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdggt\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt\") pod \"e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.413876 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.413802 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:23.552270 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.552240 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:42:23.553725 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:42:23.553699 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2167028_07f9_4203_936b_a0325c77b25d.slice/crio-841314b706820a554ec307dfa23efd1f2e728c8748c0b966b141e7c78608ea55 WatchSource:0}: Error finding container 841314b706820a554ec307dfa23efd1f2e728c8748c0b966b141e7c78608ea55: Status 404 returned error can't find the container with id 841314b706820a554ec307dfa23efd1f2e728c8748c0b966b141e7c78608ea55 Jul 30 21:42:23.556045 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.556006 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:23.556122 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.556102 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:23.556161 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:23.556147 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:24.305285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:24.305249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerStarted","Data":"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746"} Jul 30 21:42:24.305662 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:24.305304 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerStarted","Data":"841314b706820a554ec307dfa23efd1f2e728c8748c0b966b141e7c78608ea55"} Jul 30 21:42:24.323228 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:24.323177 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podStartSLOduration=1.323161529 podStartE2EDuration="1.323161529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:42:24.321624531 +0000 UTC m=+2715.020760422" watchObservedRunningTime="2026-07-30 21:42:24.323161529 +0000 UTC m=+2715.022297480" Jul 30 21:42:24.414893 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:24.414864 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:24.419820 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:24.419792 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:25.310473 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:25.310443 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:25.311325 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:25.311305 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:27.455539 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.455514 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:42:27.521281 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521250 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521433 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521300 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521433 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521331 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521547 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521507 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521618 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521569 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521618 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:27.521678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521630 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:27.521678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521653 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521699 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521753 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521801 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlt7j\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j\") pod \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\" (UID: \"c6cb5e89-cc45-4bfb-8791-59b99fd77dff\") " Jul 30 21:42:27.521835 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521798 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:27.522013 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.521900 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:27.522162 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.522138 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.522285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.522165 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.522285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.522180 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.522285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.522192 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.522285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.522161 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:42:27.523969 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.523940 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:42:27.524069 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.524022 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:42:27.524069 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.524054 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:42:27.524160 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.524093 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j" (OuterVolumeSpecName: "kube-api-access-qlt7j") pod "c6cb5e89-cc45-4bfb-8791-59b99fd77dff" (UID: "c6cb5e89-cc45-4bfb-8791-59b99fd77dff"). InnerVolumeSpecName "kube-api-access-qlt7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:42:27.622902 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.622868 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.622902 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.622898 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.622902 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.622909 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.623129 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.622921 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlt7j\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-kube-api-access-qlt7j\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:27.623129 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:27.622936 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6cb5e89-cc45-4bfb-8791-59b99fd77dff-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:42:28.327814 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.327777 2567 generic.go:356] "Generic (PLEG): container finished" podID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerID="dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b" exitCode=0 Jul 30 21:42:28.327976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.327832 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" Jul 30 21:42:28.327976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.327866 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerDied","Data":"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b"} Jul 30 21:42:28.327976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.327923 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s" event={"ID":"c6cb5e89-cc45-4bfb-8791-59b99fd77dff","Type":"ContainerDied","Data":"0b739fe73d5a129c76224238f22ffd80f90b1e78e496c52215d479ec893a98eb"} Jul 30 21:42:28.327976 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.327954 2567 scope.go:117] "RemoveContainer" containerID="dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b" Jul 30 21:42:28.337370 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.337349 2567 scope.go:117] "RemoveContainer" containerID="d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419" Jul 30 21:42:28.344154 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.344097 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:42:28.346494 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.346474 2567 scope.go:117] "RemoveContainer" containerID="dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b" Jul 30 21:42:28.346841 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:42:28.346821 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b\": container with ID starting with dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b not found: ID does not exist" containerID="dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b" Jul 30 21:42:28.346923 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.346853 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b"} err="failed to get container status \"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b\": rpc error: code = NotFound desc = could not find container \"dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b\": container with ID starting with dbd08a932045e4c2ad36140839d5bc3fbeb5cde9e771ae9ab5e0e5be72b8361b not found: ID does not exist" Jul 30 21:42:28.346923 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.346879 2567 scope.go:117] "RemoveContainer" containerID="d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419" Jul 30 21:42:28.347155 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:42:28.347132 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419\": container with ID starting with d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419 not found: ID does not exist" containerID="d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419" Jul 30 21:42:28.347197 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.347165 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419"} err="failed to get container status \"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419\": rpc error: code = NotFound desc = could not find container \"d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419\": container with ID starting with d1042ff8af1c3351bed229635a77c30c115fba82dd48745525c3955abf1ee419 not found: ID does not exist" Jul 30 21:42:28.347719 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:28.347703 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d8e5fa-openshift-default-5758b68876-wjq8s"] Jul 30 21:42:29.758805 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:29.758775 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" path="/var/lib/kubelet/pods/c6cb5e89-cc45-4bfb-8791-59b99fd77dff/volumes" Jul 30 21:42:40.312246 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.312208 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:40.312687 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.312275 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:40.397261 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.397228 2567 generic.go:356] "Generic (PLEG): container finished" podID="b2167028-07f9-4203-936b-a0325c77b25d" containerID="fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746" exitCode=0 Jul 30 21:42:40.397407 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.397299 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerDied","Data":"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746"} Jul 30 21:42:40.397720 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.397701 2567 scope.go:117] "RemoveContainer" containerID="fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746" Jul 30 21:42:40.398750 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.398711 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:40.398849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.398776 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:40.398849 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:40.398811 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:41.409905 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:41.409858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerStarted","Data":"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d"} Jul 30 21:42:41.414361 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:41.414325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:41.414538 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:41.414433 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:41.414538 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:41.414485 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:42.414866 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:42.414815 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:42.415307 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:42.414890 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:43.414897 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:43.414862 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:43.415391 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:43.414926 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:44.415192 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:44.415157 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:44.415569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:44.415209 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:45.414700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:45.414660 2567 patch_prober.go:28] interesting pod/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 30 21:42:45.414914 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:45.414716 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 30 21:42:46.420044 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:46.420008 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:46.420529 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:46.420378 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:46.421196 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:46.421166 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:42:51.844701 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.844666 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:42:51.845219 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845203 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:42:51.845278 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845222 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:42:51.845278 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845255 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:51.845278 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845265 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd96194-3806-42d5-86aa-8573ccbe11d8" containerName="istio-proxy" Jul 30 21:42:51.845378 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845350 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:42:51.845378 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.845362 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:42:51.848764 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.848743 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.850565 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.850546 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-592c1e-openshift-default-dockercfg-4rz5p\"" Jul 30 21:42:51.859324 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.859299 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:42:51.930311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930284 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmh5h\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930321 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930396 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930449 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930482 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:51.930659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:51.930597 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031053 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031096 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmh5h\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031159 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031193 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031452 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031334 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.031894 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.031865 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.032130 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.032075 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.032249 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.032154 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.032321 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.032268 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.032468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.032446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.034159 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.034135 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.034255 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.034162 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.038464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.038443 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.038555 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.038488 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmh5h\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h\") pod \"e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.163199 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.163118 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:52.299114 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.299086 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:42:52.300362 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:42:52.300335 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa858244_77c6_4515_a624_db68a433f327.slice/crio-cca4dae61e486987214c59d817546a05fe45b6e9b2472625b14c11e26796a421 WatchSource:0}: Error finding container cca4dae61e486987214c59d817546a05fe45b6e9b2472625b14c11e26796a421: Status 404 returned error can't find the container with id cca4dae61e486987214c59d817546a05fe45b6e9b2472625b14c11e26796a421 Jul 30 21:42:52.302982 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.302945 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:52.303082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.303021 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:52.303082 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.303050 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:42:52.474955 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.474870 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerStarted","Data":"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85"} Jul 30 21:42:52.474955 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.474928 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerStarted","Data":"cca4dae61e486987214c59d817546a05fe45b6e9b2472625b14c11e26796a421"} Jul 30 21:42:52.493564 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:52.493516 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podStartSLOduration=1.49350156 podStartE2EDuration="1.49350156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:42:52.492320756 +0000 UTC m=+2743.191456650" watchObservedRunningTime="2026-07-30 21:42:52.49350156 +0000 UTC m=+2743.192637451" Jul 30 21:42:53.163968 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:53.163935 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:53.165497 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:53.165476 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:42:53.165569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:53.165545 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:42:54.167732 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:54.167700 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:54.487001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:54.486922 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:42:54.487989 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:42:54.487973 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:43:08.564721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.564687 2567 generic.go:356] "Generic (PLEG): container finished" podID="aa858244-77c6-4515-a624-db68a433f327" containerID="d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85" exitCode=0 Jul 30 21:43:08.565179 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.564763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerDied","Data":"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85"} Jul 30 21:43:08.565179 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.565162 2567 scope.go:117] "RemoveContainer" containerID="d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85" Jul 30 21:43:08.566180 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.566148 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:43:08.566280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.566227 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:43:08.566280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:08.566261 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:43:09.487241 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:09.487205 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:43:09.571890 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:09.571858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerStarted","Data":"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333"} Jul 30 21:43:10.163383 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:10.163344 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:43:10.164612 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:10.164564 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:43:10.164752 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:10.164650 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:43:11.164342 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:11.164305 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:43:11.164765 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:11.164372 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:43:12.164442 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:12.164402 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:43:12.164883 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:12.164458 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:43:13.164189 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:13.164151 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:43:13.164381 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:13.164206 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:43:14.163953 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:14.163914 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 30 21:43:14.164340 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:14.163970 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 30 21:43:15.167654 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:15.167625 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:43:15.168137 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:15.167956 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:43:15.168531 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:43:15.168510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:45:00.141159 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.141125 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb"] Jul 30 21:45:00.142056 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.142025 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:45:00.142056 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.142055 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6cb5e89-cc45-4bfb-8791-59b99fd77dff" containerName="istio-proxy" Jul 30 21:45:00.145365 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.145345 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" Jul 30 21:45:00.147604 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.147561 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pwwf4\"" Jul 30 21:45:00.153293 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.153264 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb"] Jul 30 21:45:00.157773 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.157753 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" Jul 30 21:45:00.296114 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:00.296080 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb"] Jul 30 21:45:00.297837 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:45:00.297811 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3c91ef_c91b_489e_a928_6fb0c69ca81e.slice/crio-bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001 WatchSource:0}: Error finding container bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001: Status 404 returned error can't find the container with id bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001 Jul 30 21:45:01.197797 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:01.197756 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" event={"ID":"3f3c91ef-c91b-489e-a928-6fb0c69ca81e","Type":"ContainerStarted","Data":"b6a2a8ff0800e35be149a08f7b701d6df0aa751a7088ea8b5e8b5e136d6355bd"} Jul 30 21:45:01.198125 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:01.197869 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" event={"ID":"3f3c91ef-c91b-489e-a928-6fb0c69ca81e","Type":"ContainerStarted","Data":"bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001"} Jul 30 21:45:01.212397 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:01.212352 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" podStartSLOduration=1.21233644 podStartE2EDuration="1.21233644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:45:01.209288708 +0000 UTC m=+2871.908424592" watchObservedRunningTime="2026-07-30 21:45:01.21233644 +0000 UTC m=+2871.911472330" Jul 30 21:45:02.204793 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:02.204703 2567 generic.go:356] "Generic (PLEG): container finished" podID="3f3c91ef-c91b-489e-a928-6fb0c69ca81e" containerID="b6a2a8ff0800e35be149a08f7b701d6df0aa751a7088ea8b5e8b5e136d6355bd" exitCode=0 Jul 30 21:45:02.205164 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:02.204797 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" event={"ID":"3f3c91ef-c91b-489e-a928-6fb0c69ca81e","Type":"ContainerDied","Data":"b6a2a8ff0800e35be149a08f7b701d6df0aa751a7088ea8b5e8b5e136d6355bd"} Jul 30 21:45:03.344291 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:03.344267 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" Jul 30 21:45:04.217015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:04.216939 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" Jul 30 21:45:04.217015 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:04.216974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757465-6vkdb" event={"ID":"3f3c91ef-c91b-489e-a928-6fb0c69ca81e","Type":"ContainerDied","Data":"bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001"} Jul 30 21:45:04.217205 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:45:04.217021 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd9cd21a0e5e0af7e6a86c236d2ac1547c093cd5298482cb1faa42d967e94001" Jul 30 21:46:10.111676 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:46:10.111638 2567 scope.go:117] "RemoveContainer" containerID="55b964410040cf88f5d286ff105afc3967e71daf54339dda5e0f8548f78b1f4d" Jul 30 21:47:10.004941 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:47:10.004868 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:47:10.014616 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:47:10.014591 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:52:10.042164 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:10.042132 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:52:10.053348 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:10.053320 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:52:34.679412 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.679328 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:52:34.680001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.679759 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3c91ef-c91b-489e-a928-6fb0c69ca81e" containerName="cleanup" Jul 30 21:52:34.680001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.679773 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3c91ef-c91b-489e-a928-6fb0c69ca81e" containerName="cleanup" Jul 30 21:52:34.680001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.679865 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3c91ef-c91b-489e-a928-6fb0c69ca81e" containerName="cleanup" Jul 30 21:52:34.682918 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.682900 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.685081 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.685064 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-df94bf-openshift-default-dockercfg-fznl4\"" Jul 30 21:52:34.692761 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.692739 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:52:34.790315 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790283 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790315 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790316 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlf2t\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790557 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790338 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790557 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790428 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790557 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790488 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790557 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790768 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790647 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790768 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.790768 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.790711 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.891876 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.891840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.891876 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.891885 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlf2t\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.891917 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.891964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892542 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892542 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892542 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892528 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892721 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.892927 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.892904 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.894443 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.894424 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.894696 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.894678 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.899967 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.899943 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlf2t\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.900061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.899969 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token\") pod \"e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:34.994713 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:34.994638 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:35.129888 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.129864 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:52:35.131895 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:52:35.131869 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e16867_a33b_4371_9ad4_74ec41be23d8.slice/crio-4e1550bdc58845adfb9a04d6a0f72eeda0c14ccc863943b845314ddd4b74aaf4 WatchSource:0}: Error finding container 4e1550bdc58845adfb9a04d6a0f72eeda0c14ccc863943b845314ddd4b74aaf4: Status 404 returned error can't find the container with id 4e1550bdc58845adfb9a04d6a0f72eeda0c14ccc863943b845314ddd4b74aaf4 Jul 30 21:52:35.134032 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.134000 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:35.134123 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.134063 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:35.134123 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.134091 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:35.707346 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.707313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerStarted","Data":"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e"} Jul 30 21:52:35.707878 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.707360 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerStarted","Data":"4e1550bdc58845adfb9a04d6a0f72eeda0c14ccc863943b845314ddd4b74aaf4"} Jul 30 21:52:35.726106 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.726027 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podStartSLOduration=1.726007916 podStartE2EDuration="1.726007916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:52:35.725224057 +0000 UTC m=+3326.424359944" watchObservedRunningTime="2026-07-30 21:52:35.726007916 +0000 UTC m=+3326.425143807" Jul 30 21:52:35.995264 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:35.995184 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:37.000674 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:37.000646 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:37.725524 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:37.725483 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:37.726530 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:37.726510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:51.805285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.805252 2567 generic.go:356] "Generic (PLEG): container finished" podID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerID="c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e" exitCode=0 Jul 30 21:52:51.805779 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.805341 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerDied","Data":"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e"} Jul 30 21:52:51.805869 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.805849 2567 scope.go:117] "RemoveContainer" containerID="c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e" Jul 30 21:52:51.807152 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.807118 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:51.807254 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.807205 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:51.807254 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:51.807240 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:52:52.725707 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:52.725664 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:52.817387 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:52.817348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerStarted","Data":"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9"} Jul 30 21:52:52.995872 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:52.995781 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:52.997679 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:52.997285 2567 patch_prober.go:28] interesting pod/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 30 21:52:52.997679 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:52.997352 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 30 21:52:53.995445 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:53.995398 2567 patch_prober.go:28] interesting pod/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 30 21:52:53.995979 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:53.995467 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 30 21:52:54.995568 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:54.995529 2567 patch_prober.go:28] interesting pod/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 30 21:52:54.996080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:54.995620 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 30 21:52:55.995672 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:55.995637 2567 patch_prober.go:28] interesting pod/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 30 21:52:55.996276 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:55.996243 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 30 21:52:56.995490 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:56.995446 2567 patch_prober.go:28] interesting pod/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 30 21:52:56.995828 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:56.995794 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 30 21:52:57.999535 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:57.999503 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:57.999970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:57.999784 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:52:58.000607 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:52:58.000573 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:53:03.600640 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.600608 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:53:03.603285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.603258 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.605775 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.605753 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-b274d1-openshift-default-dockercfg-5z4dt\"" Jul 30 21:53:03.628104 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.628078 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:53:03.687054 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687193 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687064 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687193 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687193 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687158 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687218 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687311 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pstg5\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687328 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.687409 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.687369 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788656 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788656 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788665 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pstg5\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788832 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.788940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788933 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.789210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.789210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.788998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.789210 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.789062 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.789822 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.789734 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.789822 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.789740 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.790323 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.790299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.790493 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.790377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.792120 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.792076 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.792482 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.792448 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.800184 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.800157 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.801475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.801453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pstg5\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5\") pod \"e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:03.914078 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:03.913997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:04.047081 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.047056 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:53:04.049040 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:53:04.049014 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7975b340_ff9f_4ec1_b6b3_dc3581a8b7e9.slice/crio-eec1a50ac681145ad31f0e1193736d8ba304275cde791b7574d1dc10b687c0b5 WatchSource:0}: Error finding container eec1a50ac681145ad31f0e1193736d8ba304275cde791b7574d1dc10b687c0b5: Status 404 returned error can't find the container with id eec1a50ac681145ad31f0e1193736d8ba304275cde791b7574d1dc10b687c0b5 Jul 30 21:53:04.051799 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.051765 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:04.051891 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.051830 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:04.051891 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.051858 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:04.881077 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.881039 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerStarted","Data":"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28"} Jul 30 21:53:04.881611 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.881095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerStarted","Data":"eec1a50ac681145ad31f0e1193736d8ba304275cde791b7574d1dc10b687c0b5"} Jul 30 21:53:04.901027 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.900982 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podStartSLOduration=1.900968244 podStartE2EDuration="1.900968244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:53:04.899318479 +0000 UTC m=+3355.598454369" watchObservedRunningTime="2026-07-30 21:53:04.900968244 +0000 UTC m=+3355.600104188" Jul 30 21:53:04.914220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:04.914195 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:05.920038 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:05.920005 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:06.892376 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:06.892342 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:06.893351 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:06.893329 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:20.971994 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.971958 2567 generic.go:356] "Generic (PLEG): container finished" podID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerID="ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28" exitCode=0 Jul 30 21:53:20.972678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.972003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerDied","Data":"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28"} Jul 30 21:53:20.972678 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.972467 2567 scope.go:117] "RemoveContainer" containerID="ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28" Jul 30 21:53:20.973777 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.973742 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:20.973889 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.973832 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:20.973889 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:20.973870 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:53:21.893238 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:21.893181 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:21.980297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:21.980260 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerStarted","Data":"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5"} Jul 30 21:53:22.914974 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:22.914937 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:22.916050 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:22.916026 2567 patch_prober.go:28] interesting pod/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 30 21:53:22.916125 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:22.916083 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 30 21:53:23.915098 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:23.915056 2567 patch_prober.go:28] interesting pod/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 30 21:53:23.915500 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:23.915117 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 30 21:53:24.915320 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:24.915280 2567 patch_prober.go:28] interesting pod/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 30 21:53:24.915732 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:24.915336 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 30 21:53:25.915102 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:25.915061 2567 patch_prober.go:28] interesting pod/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 30 21:53:25.915281 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:25.915120 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 30 21:53:26.918741 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:26.918706 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:26.919116 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:26.919073 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:53:26.919666 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:53:26.919646 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:54:29.490865 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.490823 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:54:29.494886 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.494861 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.498427 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.498402 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-7d83c0-openshift-default-dockercfg-f8n2h\"" Jul 30 21:54:29.504850 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.504826 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:54:29.593965 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.593936 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.593971 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxr9f\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.593991 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594076 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594092 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.594233 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.594219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695373 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxr9f\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695518 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695569 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695863 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695863 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.695863 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.695724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.696567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.696533 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.696837 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.696813 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.697251 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.697223 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.697802 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.697776 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.697937 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.697917 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.700730 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.700709 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.713600 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.711827 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.715031 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.714913 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxr9f\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.720599 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.716640 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy\") pod \"e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.808417 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.808386 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:29.947222 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.947199 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:54:29.949279 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:54:29.949252 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d175f1c_43cd_4a2c_9cc8_82f2f36f7d66.slice/crio-a12a3614d923d60621574a01cc637fb2c7b01204fae5e97c5c2b665d9f80edcb WatchSource:0}: Error finding container a12a3614d923d60621574a01cc637fb2c7b01204fae5e97c5c2b665d9f80edcb: Status 404 returned error can't find the container with id a12a3614d923d60621574a01cc637fb2c7b01204fae5e97c5c2b665d9f80edcb Jul 30 21:54:29.951589 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.951538 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:29.951677 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.951641 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:29.951719 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:29.951685 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:30.368550 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:30.368513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerStarted","Data":"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b"} Jul 30 21:54:30.368768 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:30.368607 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerStarted","Data":"a12a3614d923d60621574a01cc637fb2c7b01204fae5e97c5c2b665d9f80edcb"} Jul 30 21:54:30.388916 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:30.388874 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podStartSLOduration=1.3888612519999999 podStartE2EDuration="1.388861252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:54:30.385281297 +0000 UTC m=+3441.084417187" watchObservedRunningTime="2026-07-30 21:54:30.388861252 +0000 UTC m=+3441.087997141" Jul 30 21:54:30.808659 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:30.808631 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:31.814331 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:31.814299 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:32.381623 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:32.381557 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:32.387786 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:32.387759 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:34.092413 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.092378 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:54:34.096157 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.096135 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.098325 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.098305 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7d83c0-dockercfg-mh8gj\"" Jul 30 21:54:34.098440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.098359 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7d83c0\"" Jul 30 21:54:34.108571 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.108548 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:54:34.139488 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.139458 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nntl\" (UniqueName: \"kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.139609 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.139515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.240906 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.240870 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nntl\" (UniqueName: \"kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.241088 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.240925 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.241749 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.241732 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.247794 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.247768 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nntl\" (UniqueName: \"kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl\") pod \"payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.409176 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.409095 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:34.543846 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:34.543818 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:54:34.545812 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:54:34.545783 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54b1011_f685_466e_ba99_1457f68ac380.slice/crio-bce854aa90e36491a09e04b5522a28ba796e5ecfe3ff86fb107d4e2118fb4c7f WatchSource:0}: Error finding container bce854aa90e36491a09e04b5522a28ba796e5ecfe3ff86fb107d4e2118fb4c7f: Status 404 returned error can't find the container with id bce854aa90e36491a09e04b5522a28ba796e5ecfe3ff86fb107d4e2118fb4c7f Jul 30 21:54:35.398509 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:35.398468 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" event={"ID":"f54b1011-f685-466e-ba99-1457f68ac380","Type":"ContainerStarted","Data":"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9"} Jul 30 21:54:35.399093 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:35.398648 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:35.399093 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:35.398695 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" event={"ID":"f54b1011-f685-466e-ba99-1457f68ac380","Type":"ContainerStarted","Data":"bce854aa90e36491a09e04b5522a28ba796e5ecfe3ff86fb107d4e2118fb4c7f"} Jul 30 21:54:35.413955 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:35.413912 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" podStartSLOduration=1.413898527 podStartE2EDuration="1.413898527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:54:35.411797593 +0000 UTC m=+3446.110933482" watchObservedRunningTime="2026-07-30 21:54:35.413898527 +0000 UTC m=+3446.113034442" Jul 30 21:54:46.407312 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:46.407276 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:54:47.805065 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:47.805031 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:47006->10.133.0.61:15021: read: connection reset by peer" start-of-body= Jul 30 21:54:47.805517 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:47.805117 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": read tcp 10.133.0.2:47006->10.133.0.61:15021: read: connection reset by peer" Jul 30 21:54:48.479739 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.479705 2567 generic.go:356] "Generic (PLEG): container finished" podID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerID="97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b" exitCode=0 Jul 30 21:54:48.479926 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.479766 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerDied","Data":"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b"} Jul 30 21:54:48.480149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.480134 2567 scope.go:117] "RemoveContainer" containerID="97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b" Jul 30 21:54:48.481223 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.481187 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:48.481280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.481269 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:48.481319 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:48.481298 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 30 21:54:49.487089 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:49.487054 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerStarted","Data":"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d"} Jul 30 21:54:49.808821 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:49.808777 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:49.809110 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:49.809079 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 30 21:54:49.809220 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:49.809147 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 30 21:54:50.813076 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:50.813028 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 30 21:54:50.813622 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:50.813122 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 30 21:54:51.809205 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:51.809160 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 30 21:54:51.809492 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:51.809460 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 30 21:54:52.809574 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:52.809536 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 30 21:54:52.810161 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:52.810129 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 30 21:54:53.808871 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:53.808840 2567 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 30 21:54:53.809146 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:53.809117 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 30 21:54:54.813488 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:54.813458 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:54.813997 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:54.813866 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:54:54.814568 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:54:54.814548 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:55:04.057071 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.057038 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:55:04.057564 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.057508 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" podUID="f54b1011-f685-466e-ba99-1457f68ac380" containerName="payload-processing" containerID="cri-o://4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9" gracePeriod=30 Jul 30 21:55:04.334262 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.334234 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:55:04.456567 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.456534 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nntl\" (UniqueName: \"kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl\") pod \"f54b1011-f685-466e-ba99-1457f68ac380\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " Jul 30 21:55:04.456762 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.456657 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume\") pod \"f54b1011-f685-466e-ba99-1457f68ac380\" (UID: \"f54b1011-f685-466e-ba99-1457f68ac380\") " Jul 30 21:55:04.457037 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.457013 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f54b1011-f685-466e-ba99-1457f68ac380" (UID: "f54b1011-f685-466e-ba99-1457f68ac380"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:55:04.458561 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.458533 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl" (OuterVolumeSpecName: "kube-api-access-2nntl") pod "f54b1011-f685-466e-ba99-1457f68ac380" (UID: "f54b1011-f685-466e-ba99-1457f68ac380"). InnerVolumeSpecName "kube-api-access-2nntl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:55:04.557825 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.557772 2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54b1011-f685-466e-ba99-1457f68ac380-plugins-config-volume\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:04.557825 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.557821 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nntl\" (UniqueName: \"kubernetes.io/projected/f54b1011-f685-466e-ba99-1457f68ac380-kube-api-access-2nntl\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:04.594024 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.593953 2567 generic.go:356] "Generic (PLEG): container finished" podID="f54b1011-f685-466e-ba99-1457f68ac380" containerID="4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9" exitCode=0 Jul 30 21:55:04.594024 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.594002 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" Jul 30 21:55:04.594187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.594044 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" event={"ID":"f54b1011-f685-466e-ba99-1457f68ac380","Type":"ContainerDied","Data":"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9"} Jul 30 21:55:04.594187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.594101 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr" event={"ID":"f54b1011-f685-466e-ba99-1457f68ac380","Type":"ContainerDied","Data":"bce854aa90e36491a09e04b5522a28ba796e5ecfe3ff86fb107d4e2118fb4c7f"} Jul 30 21:55:04.594187 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.594123 2567 scope.go:117] "RemoveContainer" containerID="4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9" Jul 30 21:55:04.605023 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.605008 2567 scope.go:117] "RemoveContainer" containerID="4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9" Jul 30 21:55:04.605258 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:55:04.605240 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9\": container with ID starting with 4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9 not found: ID does not exist" containerID="4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9" Jul 30 21:55:04.605310 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.605266 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9"} err="failed to get container status \"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9\": rpc error: code = NotFound desc = could not find container \"4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9\": container with ID starting with 4904575e2841c74981acc2659349f30e7c346e5f2b91e5531e24f7fe8f0155a9 not found: ID does not exist" Jul 30 21:55:04.621352 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.621323 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:55:04.625502 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:04.625479 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7d83c0-5464f4f87d-vn4dr"] Jul 30 21:55:05.758108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:05.758075 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54b1011-f685-466e-ba99-1457f68ac380" path="/var/lib/kubelet/pods/f54b1011-f685-466e-ba99-1457f68ac380/volumes" Jul 30 21:55:10.642189 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:10.642149 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:55:10.642738 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:10.642437 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" containerID="cri-o://4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d" gracePeriod=30 Jul 30 21:55:15.902340 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:15.902311 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:55:16.086285 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086257 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086450 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086291 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086450 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086312 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxr9f\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086450 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086380 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086566 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086526 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086661 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086565 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086661 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086629 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086771 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086635 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:55:16.086771 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086706 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:55:16.086875 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086783 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086875 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086812 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token\") pod \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\" (UID: \"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66\") " Jul 30 21:55:16.086875 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086813 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:55:16.087041 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.086966 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:55:16.087102 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087051 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:55:16.087234 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087212 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.087319 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087238 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.087319 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087252 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.087319 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087266 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.087319 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.087282 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.088600 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.088555 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:55:16.088716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.088605 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f" (OuterVolumeSpecName: "kube-api-access-hxr9f") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "kube-api-access-hxr9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:55:16.088716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.088691 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:55:16.088881 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.088855 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" (UID: "3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:55:16.188705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.188662 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.188705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.188700 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.188705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.188710 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.188705 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.188718 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxr9f\" (UniqueName: \"kubernetes.io/projected/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66-kube-api-access-hxr9f\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:55:16.685683 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.685648 2567 generic.go:356] "Generic (PLEG): container finished" podID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerID="4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d" exitCode=0 Jul 30 21:55:16.685884 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.685696 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" Jul 30 21:55:16.685884 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.685736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerDied","Data":"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d"} Jul 30 21:55:16.685884 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.685792 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686" event={"ID":"3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66","Type":"ContainerDied","Data":"a12a3614d923d60621574a01cc637fb2c7b01204fae5e97c5c2b665d9f80edcb"} Jul 30 21:55:16.685884 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.685821 2567 scope.go:117] "RemoveContainer" containerID="4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d" Jul 30 21:55:16.696194 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.696174 2567 scope.go:117] "RemoveContainer" containerID="97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b" Jul 30 21:55:16.704876 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.704851 2567 scope.go:117] "RemoveContainer" containerID="4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d" Jul 30 21:55:16.705184 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:55:16.705160 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d\": container with ID starting with 4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d not found: ID does not exist" containerID="4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d" Jul 30 21:55:16.705543 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.705204 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d"} err="failed to get container status \"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d\": rpc error: code = NotFound desc = could not find container \"4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d\": container with ID starting with 4177d9e8838184cfa52f747c02a535bbe8d42397c83d84ae64592c433c2e948d not found: ID does not exist" Jul 30 21:55:16.705543 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.705229 2567 scope.go:117] "RemoveContainer" containerID="97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b" Jul 30 21:55:16.705742 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:55:16.705698 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b\": container with ID starting with 97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b not found: ID does not exist" containerID="97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b" Jul 30 21:55:16.705809 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.705734 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b"} err="failed to get container status \"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b\": rpc error: code = NotFound desc = could not find container \"97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b\": container with ID starting with 97f6ed0f8eff247eabd9108a7e27c89e9f75af1bd67ea3c4da2d55c8c16f856b not found: ID does not exist" Jul 30 21:55:16.707554 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.707526 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:55:16.711638 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:16.711615 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7d83c0-openshift-default-7b9bc88b58-n5686"] Jul 30 21:55:17.759146 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:55:17.759112 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" path="/var/lib/kubelet/pods/3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66/volumes" Jul 30 21:56:18.122488 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:18.122459 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:56:18.123074 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:18.122690 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" containerID="cri-o://3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9" gracePeriod=30 Jul 30 21:56:23.371716 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.371692 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:56:23.406754 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.406682 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.406886 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.406775 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlf2t\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.406953 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.406905 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407009 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.406965 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407065 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407011 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407065 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407024 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:23.407170 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407107 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407170 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407140 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407266 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407266 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407221 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket\") pod \"91e16867-a33b-4371-9ad4-74ec41be23d8\" (UID: \"91e16867-a33b-4371-9ad4-74ec41be23d8\") " Jul 30 21:56:23.407365 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407270 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:23.407421 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407351 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:23.407631 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407604 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:23.407760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407696 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.407760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407719 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.407760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407721 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:56:23.407760 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.407734 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.409392 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.409359 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t" (OuterVolumeSpecName: "kube-api-access-rlf2t") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "kube-api-access-rlf2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:23.409601 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.409554 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:23.409690 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.409612 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:23.409751 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.409730 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91e16867-a33b-4371-9ad4-74ec41be23d8" (UID: "91e16867-a33b-4371-9ad4-74ec41be23d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:56:23.509061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509031 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.509061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509059 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e16867-a33b-4371-9ad4-74ec41be23d8-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.509061 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509068 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.509297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509078 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.509297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509087 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlf2t\" (UniqueName: \"kubernetes.io/projected/91e16867-a33b-4371-9ad4-74ec41be23d8-kube-api-access-rlf2t\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:23.509297 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:23.509096 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e16867-a33b-4371-9ad4-74ec41be23d8-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:24.073860 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.073764 2567 generic.go:356] "Generic (PLEG): container finished" podID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerID="3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9" exitCode=0 Jul 30 21:56:24.073860 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.073820 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" Jul 30 21:56:24.073860 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.073849 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerDied","Data":"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9"} Jul 30 21:56:24.074128 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.073922 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5" event={"ID":"91e16867-a33b-4371-9ad4-74ec41be23d8","Type":"ContainerDied","Data":"4e1550bdc58845adfb9a04d6a0f72eeda0c14ccc863943b845314ddd4b74aaf4"} Jul 30 21:56:24.074128 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.073947 2567 scope.go:117] "RemoveContainer" containerID="3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9" Jul 30 21:56:24.083121 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.083102 2567 scope.go:117] "RemoveContainer" containerID="c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e" Jul 30 21:56:24.090356 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.090333 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:56:24.092123 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.092107 2567 scope.go:117] "RemoveContainer" containerID="3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9" Jul 30 21:56:24.092409 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:24.092391 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9\": container with ID starting with 3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9 not found: ID does not exist" containerID="3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9" Jul 30 21:56:24.092482 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.092415 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9"} err="failed to get container status \"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9\": rpc error: code = NotFound desc = could not find container \"3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9\": container with ID starting with 3f3670045790456379d3ec7553f290a7d24c4588a870cb4a306afc7ccd823ae9 not found: ID does not exist" Jul 30 21:56:24.092482 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.092431 2567 scope.go:117] "RemoveContainer" containerID="c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e" Jul 30 21:56:24.092719 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:24.092688 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e\": container with ID starting with c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e not found: ID does not exist" containerID="c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e" Jul 30 21:56:24.092788 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.092728 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e"} err="failed to get container status \"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e\": rpc error: code = NotFound desc = could not find container \"c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e\": container with ID starting with c47b9352e0b470ec5792c1962e0125028ffc9739750431b925f83af259c6cc3e not found: ID does not exist" Jul 30 21:56:24.093980 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:24.093961 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-df94bf-openshift-default-957c8fc8d-sj5x5"] Jul 30 21:56:25.758700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:25.758664 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" path="/var/lib/kubelet/pods/91e16867-a33b-4371-9ad4-74ec41be23d8/volumes" Jul 30 21:56:35.650442 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:35.650407 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:56:35.650930 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:35.650704 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" containerID="cri-o://36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5" gracePeriod=30 Jul 30 21:56:40.910782 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:40.910755 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:56:41.086108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086074 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pstg5\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086108 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086111 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086134 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086174 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086202 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086242 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086303 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086330 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086325 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086410 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket\") pod \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\" (UID: \"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9\") " Jul 30 21:56:41.086664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086480 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:41.086664 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086534 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:41.086841 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086732 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:41.086841 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086776 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:56:41.086954 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086859 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.086954 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086880 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.086954 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.086895 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.087114 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.087031 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:41.088468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.088440 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:56:41.088468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.088456 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:41.088700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.088466 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5" (OuterVolumeSpecName: "kube-api-access-pstg5") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "kube-api-access-pstg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:41.088700 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.088489 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" (UID: "7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:41.173821 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.173787 2567 generic.go:356] "Generic (PLEG): container finished" podID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerID="36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5" exitCode=0 Jul 30 21:56:41.174002 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.173847 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" Jul 30 21:56:41.174002 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.173874 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerDied","Data":"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5"} Jul 30 21:56:41.174002 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.173931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd" event={"ID":"7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9","Type":"ContainerDied","Data":"eec1a50ac681145ad31f0e1193736d8ba304275cde791b7574d1dc10b687c0b5"} Jul 30 21:56:41.174002 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.173953 2567 scope.go:117] "RemoveContainer" containerID="36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5" Jul 30 21:56:41.184734 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.184715 2567 scope.go:117] "RemoveContainer" containerID="ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28" Jul 30 21:56:41.188159 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188136 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.188259 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188162 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.188259 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188177 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.188259 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188189 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.188259 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188201 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pstg5\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-kube-api-access-pstg5\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.188259 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.188215 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:41.193472 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.193455 2567 scope.go:117] "RemoveContainer" containerID="36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5" Jul 30 21:56:41.193824 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:41.193804 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5\": container with ID starting with 36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5 not found: ID does not exist" containerID="36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5" Jul 30 21:56:41.193910 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.193835 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5"} err="failed to get container status \"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5\": rpc error: code = NotFound desc = could not find container \"36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5\": container with ID starting with 36ab0e2d80e376852de091784251da65ce84c877f606a5213ac068b13ff37bf5 not found: ID does not exist" Jul 30 21:56:41.193910 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.193860 2567 scope.go:117] "RemoveContainer" containerID="ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28" Jul 30 21:56:41.194129 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:41.194110 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28\": container with ID starting with ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28 not found: ID does not exist" containerID="ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28" Jul 30 21:56:41.194177 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.194135 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28"} err="failed to get container status \"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28\": rpc error: code = NotFound desc = could not find container \"ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28\": container with ID starting with ca942efe2f08beaba7264d57ffbca38a68d4f08091732ed482013e9880dbec28 not found: ID does not exist" Jul 30 21:56:41.194854 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.194836 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:56:41.198396 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.198373 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-b274d1-openshift-default-584b44c896-sn9wd"] Jul 30 21:56:41.758336 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:41.758299 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" path="/var/lib/kubelet/pods/7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9/volumes" Jul 30 21:56:53.258321 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:53.258280 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:56:53.258877 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:53.258550 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" containerID="cri-o://486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d" gracePeriod=30 Jul 30 21:56:58.511046 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.511020 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:56:58.553011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.552946 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.552978 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553011 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553002 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdggt\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553030 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553061 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553169 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553260 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553219 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553294 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553326 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert\") pod \"b2167028-07f9-4203-936b-a0325c77b25d\" (UID: \"b2167028-07f9-4203-936b-a0325c77b25d\") " Jul 30 21:56:58.553464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553363 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:58.553464 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553402 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:58.553706 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553538 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:58.553706 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553620 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:58.553805 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553760 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.553805 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553788 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.553891 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553807 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.553891 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553821 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.553891 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.553847 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:56:58.555318 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.555293 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:58.555437 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.555397 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:56:58.555505 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.555467 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:56:58.555679 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.555651 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt" (OuterVolumeSpecName: "kube-api-access-wdggt") pod "b2167028-07f9-4203-936b-a0325c77b25d" (UID: "b2167028-07f9-4203-936b-a0325c77b25d"). InnerVolumeSpecName "kube-api-access-wdggt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:56:58.655143 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.655115 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdggt\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-kube-api-access-wdggt\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.655143 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.655140 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2167028-07f9-4203-936b-a0325c77b25d-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.655143 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.655150 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2167028-07f9-4203-936b-a0325c77b25d-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.655353 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.655158 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2167028-07f9-4203-936b-a0325c77b25d-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:58.655353 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:58.655166 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2167028-07f9-4203-936b-a0325c77b25d-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:56:59.274801 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.274766 2567 generic.go:356] "Generic (PLEG): container finished" podID="b2167028-07f9-4203-936b-a0325c77b25d" containerID="486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d" exitCode=0 Jul 30 21:56:59.274970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.274810 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" Jul 30 21:56:59.274970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.274833 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerDied","Data":"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d"} Jul 30 21:56:59.274970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.274873 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj" event={"ID":"b2167028-07f9-4203-936b-a0325c77b25d","Type":"ContainerDied","Data":"841314b706820a554ec307dfa23efd1f2e728c8748c0b966b141e7c78608ea55"} Jul 30 21:56:59.274970 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.274894 2567 scope.go:117] "RemoveContainer" containerID="486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d" Jul 30 21:56:59.284426 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.284408 2567 scope.go:117] "RemoveContainer" containerID="fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746" Jul 30 21:56:59.292379 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.292321 2567 scope.go:117] "RemoveContainer" containerID="486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d" Jul 30 21:56:59.293005 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:59.292967 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d\": container with ID starting with 486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d not found: ID does not exist" containerID="486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d" Jul 30 21:56:59.293127 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.293016 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d"} err="failed to get container status \"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d\": rpc error: code = NotFound desc = could not find container \"486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d\": container with ID starting with 486cdb574b47dfb2c1a1e4cd4b25239bcb2581f23b362e6537528d9dea60911d not found: ID does not exist" Jul 30 21:56:59.293127 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.293039 2567 scope.go:117] "RemoveContainer" containerID="fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746" Jul 30 21:56:59.293340 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:56:59.293313 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746\": container with ID starting with fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746 not found: ID does not exist" containerID="fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746" Jul 30 21:56:59.293387 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.293349 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746"} err="failed to get container status \"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746\": rpc error: code = NotFound desc = could not find container \"fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746\": container with ID starting with fed78d52e723a67069c673ce5a01dc0c61cb964f2abdd1bdf0559afb949a7746 not found: ID does not exist" Jul 30 21:56:59.295364 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.295346 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:56:59.300558 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.300539 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2abe52-openshift-default-5654f5c4d-jkxsj"] Jul 30 21:56:59.758295 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:56:59.758264 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2167028-07f9-4203-936b-a0325c77b25d" path="/var/lib/kubelet/pods/b2167028-07f9-4203-936b-a0325c77b25d/volumes" Jul 30 21:57:10.080623 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:10.080572 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:57:10.100350 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:10.100320 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:57:10.711433 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:10.711394 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:57:10.711698 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:10.711653 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" containerID="cri-o://cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333" gracePeriod=30 Jul 30 21:57:15.170065 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:15.170027 2567 patch_prober.go:28] interesting pod/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 21:57:15.170437 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:15.170086 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 21:57:15.961395 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:15.961365 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:57:16.114802 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114759 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.114802 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114805 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114837 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmh5h\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114880 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114932 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.114964 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115009 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115070 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115042 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115086 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy\") pod \"aa858244-77c6-4515-a624-db68a433f327\" (UID: \"aa858244-77c6-4515-a624-db68a433f327\") " Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115115 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115205 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115243 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115281 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:57:16.115377 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115300 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:57:16.115692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115453 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-data\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.115692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115470 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-certs\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.115692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115484 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-workload-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.115692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115497 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa858244-77c6-4515-a624-db68a433f327-istiod-ca-cert\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.115692 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.115512 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-credential-socket\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.117277 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.117249 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h" (OuterVolumeSpecName: "kube-api-access-hmh5h") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "kube-api-access-hmh5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:57:16.117386 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.117282 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 21:57:16.117386 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.117278 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 21:57:16.117386 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.117365 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa858244-77c6-4515-a624-db68a433f327" (UID: "aa858244-77c6-4515-a624-db68a433f327"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 30 21:57:16.216352 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.216326 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmh5h\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-kube-api-access-hmh5h\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.216352 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.216350 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa858244-77c6-4515-a624-db68a433f327-istio-token\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.216352 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.216360 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa858244-77c6-4515-a624-db68a433f327-istio-podinfo\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.216766 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.216368 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa858244-77c6-4515-a624-db68a433f327-istio-envoy\") on node \"ip-10-0-129-162.ec2.internal\" DevicePath \"\"" Jul 30 21:57:16.371534 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.371449 2567 generic.go:356] "Generic (PLEG): container finished" podID="aa858244-77c6-4515-a624-db68a433f327" containerID="cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333" exitCode=0 Jul 30 21:57:16.371534 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.371495 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" Jul 30 21:57:16.371720 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.371535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerDied","Data":"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333"} Jul 30 21:57:16.371720 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.371609 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn" event={"ID":"aa858244-77c6-4515-a624-db68a433f327","Type":"ContainerDied","Data":"cca4dae61e486987214c59d817546a05fe45b6e9b2472625b14c11e26796a421"} Jul 30 21:57:16.371720 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.371634 2567 scope.go:117] "RemoveContainer" containerID="cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333" Jul 30 21:57:16.381032 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.381014 2567 scope.go:117] "RemoveContainer" containerID="d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85" Jul 30 21:57:16.388856 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.388839 2567 scope.go:117] "RemoveContainer" containerID="cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333" Jul 30 21:57:16.389087 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:57:16.389069 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333\": container with ID starting with cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333 not found: ID does not exist" containerID="cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333" Jul 30 21:57:16.389209 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.389095 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333"} err="failed to get container status \"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333\": rpc error: code = NotFound desc = could not find container \"cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333\": container with ID starting with cbcb0e8831cf24699b0df35a0f2237465dcd6c9db4e2581c2434778ea82de333 not found: ID does not exist" Jul 30 21:57:16.389209 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.389112 2567 scope.go:117] "RemoveContainer" containerID="d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85" Jul 30 21:57:16.389349 ip-10-0-129-162 kubenswrapper[2567]: E0730 21:57:16.389334 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85\": container with ID starting with d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85 not found: ID does not exist" containerID="d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85" Jul 30 21:57:16.389392 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.389356 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85"} err="failed to get container status \"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85\": rpc error: code = NotFound desc = could not find container \"d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85\": container with ID starting with d3c8ea1e7bd378eb384f5898455dea2a7803f61b24339d50248e3c69d6edfa85 not found: ID does not exist" Jul 30 21:57:16.393013 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.392986 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:57:16.396086 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:16.396068 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-592c1e-openshift-default-6c98cd5cb8-l9xqn"] Jul 30 21:57:17.747090 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:17.747000 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcd94768-klwvt_f066bb5f-d3ee-44fd-973d-2d864a4e932e/manager/0.log" Jul 30 21:57:17.752730 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:17.752707 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcd94768-klwvt_f066bb5f-d3ee-44fd-973d-2d864a4e932e/copy-manifests/0.log" Jul 30 21:57:17.758244 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:17.758217 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa858244-77c6-4515-a624-db68a433f327" path="/var/lib/kubelet/pods/aa858244-77c6-4515-a624-db68a433f327/volumes" Jul 30 21:57:18.239590 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:18.239560 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-9f684b49f-4d2p2_5cac8065-969f-4f4c-846d-0eb47ff4f7fe/maas-api/0.log" Jul 30 21:57:18.593937 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:18.593910 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757465-6vkdb_3f3c91ef-c91b-489e-a928-6fb0c69ca81e/cleanup/0.log" Jul 30 21:57:20.682466 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:20.682434 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn_b2e7fa39-cf0c-4f79-9c79-a796758e63c2/istio-proxy/0.log" Jul 30 21:57:20.805751 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:20.805707 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9flg9_6eb7c4d1-2cea-4b70-8775-0b2149ba16f2/discovery/0.log" Jul 30 21:57:21.161632 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:21.161599 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/1.log" Jul 30 21:57:21.283124 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:21.283085 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-w9btr_592520a6-e0a9-4634-8366-53514a34e060/payload-pre-processing/0.log" Jul 30 21:57:22.034195 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:22.034151 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dg9q6_5ecebb9f-b6be-41ff-94ed-2088dca250bc/main/0.log" Jul 30 21:57:30.987534 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:30.987473 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ptckn_53886e58-f91a-4e1f-a5a2-331f73bc8668/global-pull-secret-syncer/0.log" Jul 30 21:57:31.127511 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:31.127474 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vg98k_1232d4ba-374c-48ee-aa85-647959e07556/konnectivity-agent/0.log" Jul 30 21:57:31.148267 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:31.148233 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-162.ec2.internal_c3cca40a41c5baf92766bc73cc200c75/haproxy/0.log" Jul 30 21:57:38.188442 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.188401 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-rbnpt_1931ef5c-2749-415d-a621-3bc663c1d096/cluster-monitoring-operator/0.log" Jul 30 21:57:38.222917 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.222879 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-nqg66_9e463673-a152-4063-adba-af75c5e1b51a/kube-state-metrics/0.log" Jul 30 21:57:38.245001 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.244979 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-nqg66_9e463673-a152-4063-adba-af75c5e1b51a/kube-rbac-proxy-main/0.log" Jul 30 21:57:38.265723 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.265694 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-nqg66_9e463673-a152-4063-adba-af75c5e1b51a/kube-rbac-proxy-self/0.log" Jul 30 21:57:38.296961 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.296941 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66cfdf6f9-ckdtm_2851a591-83a0-47de-911e-01c6e52fe25a/metrics-server/0.log" Jul 30 21:57:38.478441 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.478353 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnl62_ddb0ab00-236d-45e1-896a-18270539a50b/node-exporter/0.log" Jul 30 21:57:38.509851 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.509827 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnl62_ddb0ab00-236d-45e1-896a-18270539a50b/kube-rbac-proxy/0.log" Jul 30 21:57:38.536140 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.536121 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnl62_ddb0ab00-236d-45e1-896a-18270539a50b/init-textfile/0.log" Jul 30 21:57:38.969792 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.969753 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-c764m_f61fbd71-93d9-4f66-9c27-11cda64a44fd/prometheus-operator/0.log" Jul 30 21:57:38.991779 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:38.991756 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-c764m_f61fbd71-93d9-4f66-9c27-11cda64a44fd/kube-rbac-proxy/0.log" Jul 30 21:57:39.158724 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.158696 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/thanos-query/0.log" Jul 30 21:57:39.182952 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.182906 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/kube-rbac-proxy-web/0.log" Jul 30 21:57:39.205828 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.205808 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/kube-rbac-proxy/0.log" Jul 30 21:57:39.231206 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.231152 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/prom-label-proxy/0.log" Jul 30 21:57:39.252267 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.252247 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/kube-rbac-proxy-rules/0.log" Jul 30 21:57:39.273628 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:39.273608 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-598977cc7d-hfgvx_6fe4b2a6-3641-4f4f-bb7a-5d5ac826366b/kube-rbac-proxy-metrics/0.log" Jul 30 21:57:41.295959 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.295929 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/2.log" Jul 30 21:57:41.299940 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.299918 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-cpjjc_6eb12ad6-bdc8-41a9-832f-9dcc6df1eb31/console-operator/3.log" Jul 30 21:57:41.366522 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366441 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7"] Jul 30 21:57:41.366960 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366941 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" Jul 30 21:57:41.366960 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366961 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366975 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366986 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366992 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.366998 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367012 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367017 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367024 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367029 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367036 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367041 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367050 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54b1011-f685-466e-ba99-1457f68ac380" containerName="payload-processing" Jul 30 21:57:41.367080 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367060 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54b1011-f685-466e-ba99-1457f68ac380" containerName="payload-processing" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367121 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367131 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367140 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367146 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367152 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367158 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d175f1c-43cd-4a2c-9cc8-82f2f36f7d66" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367163 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91e16867-a33b-4371-9ad4-74ec41be23d8" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367170 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2167028-07f9-4203-936b-a0325c77b25d" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367175 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7975b340-ff9f-4ec1-b6b3-dc3581a8b7e9" containerName="istio-proxy" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367181 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54b1011-f685-466e-ba99-1457f68ac380" containerName="payload-processing" Jul 30 21:57:41.367440 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.367188 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa858244-77c6-4515-a624-db68a433f327" containerName="istio-proxy" Jul 30 21:57:41.370211 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.370196 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.371938 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.371919 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ptt2n\"/\"default-dockercfg-qrlrh\"" Jul 30 21:57:41.372403 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.372387 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ptt2n\"/\"kube-root-ca.crt\"" Jul 30 21:57:41.372479 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.372411 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ptt2n\"/\"openshift-service-ca.crt\"" Jul 30 21:57:41.377235 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.377209 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7"] Jul 30 21:57:41.423710 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.423686 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqchn\" (UniqueName: \"kubernetes.io/projected/fad22a2a-9b31-4510-9605-1b305a44999a-kube-api-access-bqchn\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.423834 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.423745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-sys\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.423834 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.423794 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-lib-modules\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.423911 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.423832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-proc\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.423911 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.423849 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-podres\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524752 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-proc\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524752 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-podres\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqchn\" (UniqueName: \"kubernetes.io/projected/fad22a2a-9b31-4510-9605-1b305a44999a-kube-api-access-bqchn\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524839 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-proc\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524858 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-sys\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-lib-modules\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.524934 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-podres\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.525113 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.524981 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-sys\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.525113 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.525016 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fad22a2a-9b31-4510-9605-1b305a44999a-lib-modules\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.532111 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.532080 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqchn\" (UniqueName: \"kubernetes.io/projected/fad22a2a-9b31-4510-9605-1b305a44999a-kube-api-access-bqchn\") pod \"perf-node-gather-daemonset-bctb7\" (UID: \"fad22a2a-9b31-4510-9605-1b305a44999a\") " pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.682275 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.682173 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:41.811382 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.811353 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7"] Jul 30 21:57:41.813453 ip-10-0-129-162 kubenswrapper[2567]: W0730 21:57:41.813421 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfad22a2a_9b31_4510_9605_1b305a44999a.slice/crio-97e151fe167fa0a14e515f85b16e4f4b0ec914c054c65cbbb2213c0dba1f96c7 WatchSource:0}: Error finding container 97e151fe167fa0a14e515f85b16e4f4b0ec914c054c65cbbb2213c0dba1f96c7: Status 404 returned error can't find the container with id 97e151fe167fa0a14e515f85b16e4f4b0ec914c054c65cbbb2213c0dba1f96c7 Jul 30 21:57:41.943626 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.943542 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69d7ddf5f8-tr28w_9a27b83b-8a40-45fa-bc0e-6e5329c3b0a0/console/0.log" Jul 30 21:57:41.977957 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:41.977926 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-sz96w_3f2d4ab7-8141-415a-a91e-5554bc15aace/download-server/0.log" Jul 30 21:57:42.510149 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:42.510110 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" event={"ID":"fad22a2a-9b31-4510-9605-1b305a44999a","Type":"ContainerStarted","Data":"4f4c27ff3aa286ac990599fc0a0d239d1c18f3b1943b7cc772bbf62e6f5daa67"} Jul 30 21:57:42.510542 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:42.510168 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" event={"ID":"fad22a2a-9b31-4510-9605-1b305a44999a","Type":"ContainerStarted","Data":"97e151fe167fa0a14e515f85b16e4f4b0ec914c054c65cbbb2213c0dba1f96c7"} Jul 30 21:57:42.510542 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:42.510190 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:42.527798 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:42.527748 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" podStartSLOduration=1.527730194 podStartE2EDuration="1.527730194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 21:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:57:42.524495897 +0000 UTC m=+3633.223631798" watchObservedRunningTime="2026-07-30 21:57:42.527730194 +0000 UTC m=+3633.226866083" Jul 30 21:57:43.694349 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:43.694322 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cbq8d_20a05602-5dbd-4333-a5e1-bc095bb40f1e/dns/0.log" Jul 30 21:57:43.716065 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:43.716043 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cbq8d_20a05602-5dbd-4333-a5e1-bc095bb40f1e/kube-rbac-proxy/0.log" Jul 30 21:57:43.843458 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:43.843433 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-45ms9_8b6ea7d5-88c2-4c24-b4d3-d8fec04783c4/dns-node-resolver/0.log" Jul 30 21:57:44.477728 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:44.477689 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nk9g2_597f0712-04ff-4ba3-b59d-5df371cd7a8f/node-ca/0.log" Jul 30 21:57:45.446239 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:45.446210 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn_b2e7fa39-cf0c-4f79-9c79-a796758e63c2/istio-proxy/0.log" Jul 30 21:57:45.616903 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:45.616874 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9flg9_6eb7c4d1-2cea-4b70-8775-0b2149ba16f2/discovery/0.log" Jul 30 21:57:45.690388 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:45.690357 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/0.log" Jul 30 21:57:45.696125 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:45.696105 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/1.log" Jul 30 21:57:45.724475 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:45.724411 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-w9btr_592520a6-e0a9-4634-8366-53514a34e060/payload-pre-processing/0.log" Jul 30 21:57:46.601325 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:46.601294 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmdcg_3369fd3e-90aa-4e83-a9c5-b10b171b7d36/serve-healthcheck-canary/0.log" Jul 30 21:57:47.217056 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:47.217025 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g52jz_20ef0cf3-01e8-4b97-a573-f3ee67b25dbe/kube-rbac-proxy/0.log" Jul 30 21:57:47.240430 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:47.240405 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g52jz_20ef0cf3-01e8-4b97-a573-f3ee67b25dbe/exporter/0.log" Jul 30 21:57:47.261780 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:47.261756 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g52jz_20ef0cf3-01e8-4b97-a573-f3ee67b25dbe/extractor/0.log" Jul 30 21:57:48.525277 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:48.525250 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ptt2n/perf-node-gather-daemonset-bctb7" Jul 30 21:57:50.618925 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:50.618891 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcd94768-klwvt_f066bb5f-d3ee-44fd-973d-2d864a4e932e/manager/0.log" Jul 30 21:57:50.641282 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:50.641256 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcd94768-klwvt_f066bb5f-d3ee-44fd-973d-2d864a4e932e/copy-manifests/0.log" Jul 30 21:57:52.116214 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.116146 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76d7f64c45-rvlb9_7ad83382-33cb-4dc1-a244-c85fbd8e4264/manager/0.log" Jul 30 21:57:52.146249 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.146227 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jz2wl_c9b8f1bc-ca86-44d6-a0f7-aea7d8c4ed6b/openshift-lws-operator/0.log" Jul 30 21:57:52.237468 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.237437 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn_b2e7fa39-cf0c-4f79-9c79-a796758e63c2/istio-proxy/0.log" Jul 30 21:57:52.399898 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.399817 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9flg9_6eb7c4d1-2cea-4b70-8775-0b2149ba16f2/discovery/0.log" Jul 30 21:57:52.458091 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.458054 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/0.log" Jul 30 21:57:52.464900 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.464878 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/1.log" Jul 30 21:57:52.486280 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:52.486255 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-w9btr_592520a6-e0a9-4634-8366-53514a34e060/payload-pre-processing/0.log" Jul 30 21:57:53.916198 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:53.916164 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qs7hn_b2e7fa39-cf0c-4f79-9c79-a796758e63c2/istio-proxy/0.log" Jul 30 21:57:54.078200 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:54.078112 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9flg9_6eb7c4d1-2cea-4b70-8775-0b2149ba16f2/discovery/0.log" Jul 30 21:57:54.132527 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:54.132494 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/0.log" Jul 30 21:57:54.138830 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:54.138809 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-s8667_00fcefcc-2f59-405f-96a3-da5265867814/istio-proxy/1.log" Jul 30 21:57:54.162276 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:54.162254 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-w9btr_592520a6-e0a9-4634-8366-53514a34e060/payload-pre-processing/0.log" Jul 30 21:57:57.287769 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:57.287737 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-jp4mz_c03ee922-4a2a-4ffb-a540-e0bec4d4e77c/migrator/0.log" Jul 30 21:57:57.320888 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:57.320851 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-jp4mz_c03ee922-4a2a-4ffb-a540-e0bec4d4e77c/graceful-termination/0.log" Jul 30 21:57:58.746510 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:58.746481 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-84p8s_3db6b84c-1324-41f9-a09a-a73131477031/kube-multus/0.log" Jul 30 21:57:59.086647 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.086617 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/kube-multus-additional-cni-plugins/0.log" Jul 30 21:57:59.108848 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.108818 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/egress-router-binary-copy/0.log" Jul 30 21:57:59.129626 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.129603 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/cni-plugins/0.log" Jul 30 21:57:59.153491 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.153470 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/bond-cni-plugin/0.log" Jul 30 21:57:59.174826 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.174803 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/routeoverride-cni/0.log" Jul 30 21:57:59.197512 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.197487 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/whereabouts-cni-bincopy/0.log" Jul 30 21:57:59.219288 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.219266 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjfqt_2a37f37e-1a22-459b-ad85-aa6869c43d5c/whereabouts-cni/0.log" Jul 30 21:57:59.434158 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.434083 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qct22_f8ad8626-c218-4c3f-918d-4295881c08e3/network-metrics-daemon/0.log" Jul 30 21:57:59.455907 ip-10-0-129-162 kubenswrapper[2567]: I0730 21:57:59.455881 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qct22_f8ad8626-c218-4c3f-918d-4295881c08e3/kube-rbac-proxy/0.log"