Jul 28 11:28:48.748924 ip-10-0-130-140 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 11:28:48.748937 ip-10-0-130-140 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 11:28:48.748944 ip-10-0-130-140 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 11:28:48.749247 ip-10-0-130-140 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 11:28:58.924834 ip-10-0-130-140 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 11:28:58.924852 ip-10-0-130-140 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 709fdc3abfa44b90a245b437959640a6 --
Jul 28 11:30:35.416188 ip-10-0-130-140 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 11:30:35.857683 ip-10-0-130-140 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 11:30:35.857683 ip-10-0-130-140 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 11:30:35.857683 ip-10-0-130-140 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 11:30:35.857683 ip-10-0-130-140 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 11:30:35.857683 ip-10-0-130-140 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 11:30:35.858454 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.857771    2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 11:30:35.862736 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862720    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.862736 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862735    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862739    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862744    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862748    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862750    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862754    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862756    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862759    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862762    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862765    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862767    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862772    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862776    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862779    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862782    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862784    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862787    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862791    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862793    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.862799 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862797    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862799    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862803    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862805    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862808    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862811    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862814    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862816    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862819    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862821    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862825    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862828    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862831    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862834    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862837    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862839    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862842    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862844    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.863257 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862847    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862849    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862853    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862855    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862858    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862861    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862863    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862865    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862868    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862870    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862873    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862875    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862877    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862881    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862885    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862888    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862909    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862912    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862914    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.863687 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862917    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862919    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862922    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862925    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862928    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862930    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862932    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862935    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862937    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862940    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862943    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862945    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862948    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862950    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862953    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862955    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862958    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862960    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862963    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862965    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.864172 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862968    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862971    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862973    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862976    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862979    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862982    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862985    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862987    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862991    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862994    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862997    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.862999    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863002    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863004    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863007    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863009    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863012    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863014    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863017    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863019    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.864652 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863022    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863024    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863713    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863719    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863722    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863725    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863728    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863731    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863733    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863736    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863739    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863742    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863745    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863748    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863750    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863753    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863758    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863762    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863765    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863769    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863771    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.865201 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863774    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863777    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863780    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863782    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863785    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863787    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863790    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863792    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863795    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863797    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863799    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863802    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863804    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863807    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863810    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863812    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863815    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863817    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863819    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863822    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.865722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863825    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863827    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863830    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863832    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863835    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863838    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863840    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863843    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863845    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863848    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863852    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863854    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863857    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863860    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863863    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863865    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863868    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863870    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863873    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.866221 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863875    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863878    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863880    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863883    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863885    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863888    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863890    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863906    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863910    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863914    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863917    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863919    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863922    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863925    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863927    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863930    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863932    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863935    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863938    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.866684 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863941    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863943    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863948    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863952    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863955    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863958    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863961    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863964    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863966    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863970    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863973    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863976    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863978    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863981    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863984    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863986    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863989    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863991    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863994    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.867168 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863996    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.863999    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864001    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864347    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864351    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864355    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864357    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864360    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864363    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864365    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864368    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864371    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864374    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864376    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864379    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864381    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864384    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864387    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864390    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864393    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.867623 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864396    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864398    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864401    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864404    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864406    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864409    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864411    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864415    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864417    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864420    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864424    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864427    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864429    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864432    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864434    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864437    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864439    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864441    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864444    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864446    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.868135 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864449    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864451    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864454    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864457    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864460    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864463    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864467    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864471    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864474    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864476    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864479    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864481    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864484    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864487    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864489    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864492    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864494    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864497    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864501    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.868625 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864504    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864507    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864509    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864512    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864514    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864517    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864519    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864521    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864524    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864527    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864529    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864531    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864534    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864536    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864539    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864541    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864547    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864549    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864552    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.869097 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864555    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864557    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864560    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864562    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864565    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864567    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864570    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864572    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864575    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864577    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864583    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864586    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864589    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864591    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864594    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864596    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864599    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864602    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864604    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864606    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.869548 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864609    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864612    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864614    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.864616    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864677    2552 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864683    2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864689    2552 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864699    2552 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864703    2552 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864708    2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864712    2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864716    2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864720    2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864724    2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864727    2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864730    2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864733    2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864736    2552 flags.go:64] FLAG: --cgroup-root=""
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864739    2552 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864742    2552 flags.go:64] FLAG: --client-ca-file=""
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864745    2552 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864748    2552 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864752    2552 flags.go:64] FLAG: --cluster-domain=""
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864755    2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 11:30:35.870100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864758    2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864761    2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864765    2552 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864769    2552 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864771    2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864774    2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864781    2552 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864784    2552 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864787    2552 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864790    2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864793    2552 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864796    2552 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864800    2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864803    2552 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864806    2552 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864809    2552 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864811    2552 flags.go:64] FLAG: --enable-server="true"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864816    2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864820    2552 flags.go:64] FLAG: --event-burst="100"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864823    2552 flags.go:64] FLAG: --event-qps="50"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864827    2552 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864830    2552 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864833    2552 flags.go:64] FLAG: --eviction-hard=""
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864837    2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 11:30:35.870676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864840    2552 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864842    2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864845    2552 flags.go:64] FLAG: --eviction-soft=""
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864848    2552 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864851    2552 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864854    2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864857    2552 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864859    2552 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864863    2552 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864866    2552 flags.go:64] FLAG: --feature-gates=""
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864869    2552 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864872    2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864875    2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864878    2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864881    2552 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864884    2552 flags.go:64] FLAG: --help="false"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864887    2552 flags.go:64] FLAG: --hostname-override="ip-10-0-130-140.ec2.internal"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864890    2552 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864905    2552 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864908    2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864911    2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864914    2552 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864917    2552 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 11:30:35.871270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864920    2552 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864923    2552 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864928    2552 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864931    2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864934    2552 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864938    2552 flags.go:64] FLAG: --kube-reserved=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864940    2552 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864943    2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864946    2552 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864949    2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864952    2552 flags.go:64] FLAG: --lock-file=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864955    2552 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864958    2552 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864960    2552 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864966    2552 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864968    2552 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864972    2552 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864974    2552 flags.go:64] FLAG: --logging-format="text"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864977    2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864981    2552 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864983    2552 flags.go:64] FLAG: --manifest-url=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864986    2552 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864990    2552 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864993    2552 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864997    2552 flags.go:64] FLAG: --max-pods="110"
Jul 28 11:30:35.871868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.864999    2552 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865002    2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865005    2552 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865008    2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865011    2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865014    2552 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865016    2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865023    2552 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865026    2552 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865031    2552 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865034    2552 flags.go:64] FLAG: --pod-cidr=""
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865037    2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865043    2552 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865046    2552 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865049    2552 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865052    2552 flags.go:64] FLAG: --port="10250"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865055    2552 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865058    2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09a1688453059b24b"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865061    2552 flags.go:64] FLAG: --qos-reserved=""
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865064    2552 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865067    2552 flags.go:64] FLAG: --register-node="true"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865069    2552 flags.go:64] FLAG: --register-with-taints=""
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865073    2552 flags.go:64] FLAG: --registry-burst="10"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865076    2552 flags.go:64] FLAG: --registry-qps="5"
Jul 28 11:30:35.872512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865079    2552 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865081    2552 flags.go:64] FLAG: --reserved-memory=""
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865086    2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865089    2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865092    2552 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865094    2552 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865097    2552 flags.go:64] FLAG: --runonce="false"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865100    2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865103    2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865105    2552 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865108    2552 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865111    2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865114    2552 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865117    2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865119    2552 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865122    2552 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865125    2552 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865128    2552 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865131    2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865134    2552 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865139    2552 flags.go:64] FLAG: --system-cgroups=""
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865142    2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865147    2552 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865150    2552 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865153    2552 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 11:30:35.873493 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865157    2552 flags.go:64] FLAG: --tls-min-version=""
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865159    2552 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865162    2552 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865165    2552 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865168    2552 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865170    2552 flags.go:64] FLAG: --v="2"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865174    2552 flags.go:64] FLAG: --version="false"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865178    2552 flags.go:64] FLAG: --vmodule=""
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865181    2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.865184    2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865299    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865303    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865307    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865310    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865312    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865315    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865318    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865320    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865323    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865326    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865329    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865331    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.874324 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865334    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865336    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865339    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865343    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865345    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865349    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865352    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865355    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865358    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865361    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865363    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865366    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865368    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865371    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865373    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865376    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865379    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865381    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865384    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865388    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.874866 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865392    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865395    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865398    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865401    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865404    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865407    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865410    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865412    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865415    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865418    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865420    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865423    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865426    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865429    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865431    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865436    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865438    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865442    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865444    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865447    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.875366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865449    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865452    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865454    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865457    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865459    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865462    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865464    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865468    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865472    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865474    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865477    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865480    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865483    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865486    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865489    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865492    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865494    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865497    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865500    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.875864 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865502    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865505    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865507    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865510    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865512    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865515    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865517    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865520    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865524    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865526    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865530    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865533    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865536    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865538    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865541    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865543    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865546    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865548    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865551    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865553    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.876366 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865556    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865558    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865561    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865563    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865566    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865568    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865570    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.865573    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.866398    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.874651    2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.874667    2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.874681    2552 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.874687    2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874746    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874751    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.876860 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874755    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874758    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874761    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874764    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874768    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874771    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874773    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874776    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874779    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874782    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874784    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874787    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874790    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874793    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874795    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874798    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874800    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874803    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874806    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874808    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874811    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.877251 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874813    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874816    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874818    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874821    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874823    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874827    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874832    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874836    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874839    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874842    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874845    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874848    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874850    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874853    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874856    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874858    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874861    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874864    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874866    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.877755 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874869    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874872    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874874    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874877    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874880    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874883    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874886    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874888    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874891    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874909    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874913    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874916    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874918    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874921    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874923    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874926    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874928    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874931    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874933    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.878239 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874937    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874939    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874943    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874946    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874949    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874951    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874954    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874957    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874960    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874963    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874967    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874970    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874972    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874975    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874978    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874980    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874983    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874985    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874989    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.878725 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874992    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874994    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874997    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.874999    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875002    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875004    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875007    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875011    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875014    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875017    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875020    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875022    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875025    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875029    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875032    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875034    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875037    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875040    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.879220 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875042    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.875047    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875160    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875165    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875169    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875173    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875175    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875178    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875181    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875183    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875186    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875188    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875192    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875196    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 11:30:35.879680 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875199    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875202    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875205    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875207    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875210    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875212    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875215    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875217    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875220    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875222    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875225    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875228    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875230    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875234    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875237    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875240    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875242    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875245    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875247    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875250    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 11:30:35.880069 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875252    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875255    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875258    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875261    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875263    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875266    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875268    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875271    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875274    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875276    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875278    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875281    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875283    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875286    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875289    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875291    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875294    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875296    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875299    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875302    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 11:30:35.880545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875304    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875307    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875309    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875311    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875314    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875317    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875319    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875322    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875324    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875327    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875330    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875333    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875335    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875338    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875341    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875343    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875346    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875348    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875351    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 11:30:35.881046 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875353    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875356    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875358    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875361    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875363    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875366    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875369    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875372    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875374    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875377    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875380    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875383    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875385    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875388    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875391    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875393    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875396    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875399    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875401    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875404    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 11:30:35.881501 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875406    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875409    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875411    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875414    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875416    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875419    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875421    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:35.875425    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.875430    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.875540    2552 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.878041    2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.879746    2552 server.go:1019] "Starting client certificate rotation"
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.879844    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 11:30:35.882123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.879875    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 11:30:35.883367 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.883354    2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 11:30:35.884515 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.884498    2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 11:30:35.894653 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.894636    2552 log.go:25] "Validated CRI v1 runtime API"
Jul 28 11:30:35.897078 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.897062    2552 log.go:25] "Validated CRI v1 image API"
Jul 28 11:30:35.897840 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.897826    2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 11:30:35.902720 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.902703    2552 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a0cd4b06-d414-4ec2-95a1-f9f6ae05d0dd:/dev/nvme0n1p4 ecf4eaa9-d93f-4b52-bafc-1d649c433fb1:/dev/nvme0n1p3]
Jul 28 11:30:35.902771 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.902721    2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 11:30:35.909189 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.909085    2552 manager.go:216] Machine: {Timestamp:2026-07-28 11:30:35.907642758 +0000 UTC m=+0.364677205 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101101 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23816316d692d018a636424bff5151 SystemUUID:ec238163-16d6-92d0-18a6-36424bff5151 BootID:709fdc3a-bfa4-4b90-a245-b437959640a6 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:c4:35:c6:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:c4:35:c6:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:7b:6b:06:71:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 11:30:35.909189 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.909183    2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 11:30:35.909297 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.909278    2552 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 11:30:35.913005 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.912986    2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 11:30:35.913143 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.913007    2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-140.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 11:30:35.913809 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.913796    2552 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 11:30:35.913843 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.913818    2552 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 11:30:35.913843 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.913829    2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 11:30:35.914619 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.914606    2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 11:30:35.915799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.915789    2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 11:30:35.918656 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918645    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 11:30:35.918786 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918777    2552 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 11:30:35.918821 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918805    2552 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 11:30:35.918848 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918832    2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 11:30:35.918973 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918963    2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 11:30:35.919002 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918976    2552 kubelet.go:398] "Adding apiserver pod source"
Jul 28 11:30:35.919002 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.918985    2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 11:30:35.920066 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.920050    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 11:30:35.920170 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.920072    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 11:30:35.923108 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.923087    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 11:30:35.924376 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.924357    2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 11:30:35.926951 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.926936    2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 11:30:35.927019 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.926966    2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 11:30:35.928999 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.928988    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929003    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929009    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929014    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929023    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929029    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 11:30:35.929035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929037    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 11:30:35.929200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929042    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 11:30:35.929200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929049    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 11:30:35.929200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929055    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 11:30:35.929200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929063    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 11:30:35.929200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.929069    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 11:30:35.930938 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.930925    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 11:30:35.930938 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.930938    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 11:30:35.931753 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.931728    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 11:30:35.932069 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.932049    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 11:30:35.934085 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.934072    2552 server.go:1268] "Started kubelet"
Jul 28 11:30:35.934199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.934170    2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 11:30:35.934411 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.934341    2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 11:30:35.934498 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.934443    2552 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 11:30:35.934855 ip-10-0-130-140 systemd[1]: Started Kubernetes Kubelet.
Jul 28 11:30:35.936619 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.936505    2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 11:30:35.937058 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.937044    2552 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 11:30:35.939571 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.939534    2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 11:30:35.946145 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.946128    2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-140.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 11:30:35.947214 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.946014    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78c4b324e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.93404475 +0000 UTC m=+0.391079197,LastTimestamp:2026-07-28 11:30:35.93404475 +0000 UTC m=+0.391079197,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:35.947214 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.947141    2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 11:30:35.947214 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.947157    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 11:30:35.947380 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.947243    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:35.947930 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.947888    2552 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 11:30:35.947930 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.947922    2552 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 11:30:35.948068 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.947993    2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 11:30:35.948068 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.948056    2552 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 11:30:35.948068 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.948062    2552 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 11:30:35.948844 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.948701    2552 factory.go:55] Registering systemd factory
Jul 28 11:30:35.948844 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.948746    2552 factory.go:223] Registration of the systemd container factory successfully
Jul 28 11:30:35.949170 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949153    2552 factory.go:153] Registering CRI-O factory
Jul 28 11:30:35.949241 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949173    2552 factory.go:223] Registration of the crio container factory successfully
Jul 28 11:30:35.949279 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949246    2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 11:30:35.949279 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949266    2552 factory.go:103] Registering Raw factory
Jul 28 11:30:35.949354 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949296    2552 manager.go:1196] Started watching for new ooms in manager
Jul 28 11:30:35.949731 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.949714    2552 manager.go:349] Starting recovery of all containers
Jul 28 11:30:35.962632 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.962612    2552 manager.go:354] Recovery completed
Jul 28 11:30:35.964201 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.964179    2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 11:30:35.964606 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.964585    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 11:30:35.967866 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.967849    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:35.971528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.971499    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:35.971604 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.971539    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:35.971604 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.971553    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:35.972171 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.972158    2552 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 11:30:35.972171 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.972169    2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 11:30:35.972171 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.972171    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:35.972327 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.972184    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 11:30:35.973558 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.973540    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:35.973616 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.973571    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:35.973616 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.973586    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:35.974371 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.974295    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e8723fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971527676 +0000 UTC m=+0.428562126,LastTimestamp:2026-07-28 11:30:35.971527676 +0000 UTC m=+0.428562126,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:35.975032 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.975017    2552 policy_none.go:49] "None policy: Start"
Jul 28 11:30:35.975102 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.975035    2552 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 11:30:35.975102 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.975049    2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 11:30:35.976994 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:35.976980    2552 policy_none.go:47] "Start"
Jul 28 11:30:35.989531 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.989453    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e876cb8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971546296 +0000 UTC m=+0.428580748,LastTimestamp:2026-07-28 11:30:35.971546296 +0000 UTC m=+0.428580748,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:35.992003 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.991937    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e879f73  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971559283 +0000 UTC m=+0.428593735,LastTimestamp:2026-07-28 11:30:35.971559283 +0000 UTC m=+0.428593735,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:35.999997 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:35.999908    2552 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-140.ec2.internal.18c670b78e8723fc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e8723fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971527676 +0000 UTC m=+0.428562126,LastTimestamp:2026-07-28 11:30:35.973557419 +0000 UTC m=+0.430591874,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:36.005300 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.005242    2552 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-140.ec2.internal.18c670b78e876cb8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e876cb8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971546296 +0000 UTC m=+0.428580748,LastTimestamp:2026-07-28 11:30:35.973578121 +0000 UTC m=+0.430612574,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:36.011264 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.011172    2552 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-140.ec2.internal.18c670b78e879f73\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b78e879f73  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-140.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:35.971559283 +0000 UTC m=+0.428593735,LastTimestamp:2026-07-28 11:30:35.973591971 +0000 UTC m=+0.430626423,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:36.012473 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012458    2552 manager.go:337] "Starting Device Plugin manager"
Jul 28 11:30:36.012540 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.012495    2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 11:30:36.012540 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012509    2552 server.go:87] "Starting device plugin registration server"
Jul 28 11:30:36.012787 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012775    2552 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 11:30:36.012843 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012788    2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 11:30:36.012912 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012879    2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 11:30:36.012991 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012979    2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 11:30:36.013033 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.012994    2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 11:30:36.013815 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.013706    2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 11:30:36.013815 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.013778    2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.026668 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.026588    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-140.ec2.internal.18c670b791ad1b09  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-140.ec2.internal,UID:ip-10-0-130-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-130-140.ec2.internal,},FirstTimestamp:2026-07-28 11:30:36.024347401 +0000 UTC m=+0.481381837,LastTimestamp:2026-07-28 11:30:36.024347401 +0000 UTC m=+0.481381837,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-140.ec2.internal,}"
Jul 28 11:30:36.041711 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.028359    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ndt8z"
Jul 28 11:30:36.041711 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.037490    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ndt8z"
Jul 28 11:30:36.100973 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.100943    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 11:30:36.102172 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.102152    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 11:30:36.102263 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.102186    2552 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 11:30:36.102263 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.102208    2552 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 11:30:36.102263 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.102250    2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 11:30:36.113515 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.113342    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:36.114149 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.114132    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:36.114233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.114164    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:36.114233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.114179    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:36.114233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.114208    2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.123121 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.123101    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:36.138216 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.138200    2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.138287 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.138221    2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-140.ec2.internal\": node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.190286 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.190257    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.203202 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.203170    2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"]
Jul 28 11:30:36.203281 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.203238    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:36.204930 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.204915    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:36.205001 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.204939    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:36.205001 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.204954    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:36.207309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.207298    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:36.207426 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.207411    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.207464 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.207446    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:36.207961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.207948    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:36.208024 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.207979    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:36.208024 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.208001    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:36.208107 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.208086    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:36.208152 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.208108    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:36.208152 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.208120    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:36.210311 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.210298    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.210357 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.210322    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:36.210958 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.210941    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:36.211055 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.210966    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:36.211055 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.210979    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:36.238524 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.238508    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-140.ec2.internal\" not found" node="ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.242611 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.242596    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-140.ec2.internal\" not found" node="ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.290685 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.290656    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.348919 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.348874    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a16892ee32f85011fc1a1806c8666fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-140.ec2.internal\" (UID: \"a16892ee32f85011fc1a1806c8666fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.348919 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.348918    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.349087 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.348938    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.390801 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.390736    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.449226 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449196    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.449338 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449230    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.449338 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449286    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.449338 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449330    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a16892ee32f85011fc1a1806c8666fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-140.ec2.internal\" (UID: \"a16892ee32f85011fc1a1806c8666fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.449470 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449360    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a16892ee32f85011fc1a1806c8666fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-140.ec2.internal\" (UID: \"a16892ee32f85011fc1a1806c8666fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.449470 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.449390    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5742a08682b14dec412b7167019da7bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal\" (UID: \"5742a08682b14dec412b7167019da7bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.491301 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.491265    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.541472 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.541448    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.546683 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.546606    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:36.592107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.592082    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.695267 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.695186    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.768133 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.768101    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:36.795931 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.795888    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.879021 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.878983    2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 11:30:36.879473 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.879147    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 11:30:36.879473 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:36.879169    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 11:30:36.896981 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.896952    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:36.997778 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:36.997708    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:37.043454 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.043415    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 11:25:36 +0000 UTC" deadline="2028-04-04 22:52:36.32922812 +0000 UTC"
Jul 28 11:30:37.043454 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.043452    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14795h21m59.285780001s"
Jul 28 11:30:37.098314 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:37.098280    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-140.ec2.internal\" not found"
Jul 28 11:30:37.109328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.109296    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 11:30:37.110317 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.110299    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 11:30:37.110443 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.110331    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 11:30:37.110443 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.110346    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeHasSufficientPID"
Jul 28 11:30:37.177871 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.177844    2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:37.209259 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:37.209231    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16892ee32f85011fc1a1806c8666fbf.slice/crio-e89d96191d158a9252d1e8c508b504b7677bdac040e5a6510c2c40a7d2ee6d85 WatchSource:0}: Error finding container e89d96191d158a9252d1e8c508b504b7677bdac040e5a6510c2c40a7d2ee6d85: Status 404 returned error can't find the container with id e89d96191d158a9252d1e8c508b504b7677bdac040e5a6510c2c40a7d2ee6d85
Jul 28 11:30:37.209759 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:37.209736    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5742a08682b14dec412b7167019da7bd.slice/crio-9e353404268e2bd1b69f9b6f4cfceff2e2924a11b7fdc3368cbd0bf6de165082 WatchSource:0}: Error finding container 9e353404268e2bd1b69f9b6f4cfceff2e2924a11b7fdc3368cbd0bf6de165082: Status 404 returned error can't find the container with id 9e353404268e2bd1b69f9b6f4cfceff2e2924a11b7fdc3368cbd0bf6de165082
Jul 28 11:30:37.215480 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.215464    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:30:37.248677 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.248616    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:37.282821 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.282800    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 11:30:37.283719 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.283702    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal"
Jul 28 11:30:37.301812 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.301793    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 11:30:37.465451 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.465421    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:37.842484 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.842450    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:37.921653 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.921404    2552 apiserver.go:52] "Watching apiserver"
Jul 28 11:30:37.933114 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.933057    2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 11:30:37.933585 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.933562    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mm7q2","openshift-image-registry/node-ca-r4qzt","openshift-network-diagnostics/network-check-target-2dpkp","openshift-ovn-kubernetes/ovnkube-node-vnt2w","kube-system/konnectivity-agent-st4fs","openshift-dns/node-resolver-74fz4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal","openshift-multus/multus-additional-cni-plugins-zf9gj","openshift-multus/multus-x929n","openshift-multus/network-metrics-daemon-dqwqh","openshift-network-operator/iptables-alerter-m8j8l","kube-system/global-pull-secret-syncer-rsxxg","kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"]
Jul 28 11:30:37.936170 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.936148    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:37.936255 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:37.936228    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:37.938492 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.938455    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:37.940960 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.940620    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:37.940960 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.940674    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l2gf5\""
Jul 28 11:30:37.940960 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:37.940685    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:37.940960 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.940728    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.940960 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.940795    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 11:30:37.942269 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.942251    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.944676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.943208    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.946171 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.945500    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:37.947911 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.947877    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 11:30:37.948119 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.948104    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 11:30:37.948598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.948371    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 11:30:37.948773 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.948644    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 11:30:37.949458 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.949439    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:37.951618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.950531    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.951618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.950911    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.951618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.950982    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6tms9\""
Jul 28 11:30:37.951618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.951164    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.951618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.951258    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.955956    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jtfnw\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.956178    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.957398    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.957689    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jgqgc\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.957950    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tj8n2\""
Jul 28 11:30:37.961702 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.958116    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962231    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-k8s-cni-cncf-io\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962268    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-hostroot\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962303    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbf518f8-7555-4842-8520-a6c58b5aa6f4-iptables-alerter-script\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962369    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962406    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-netns\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962419    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962442    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-etc-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962474    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-bin\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962505    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovn-node-metrics-cert\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962719    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962754    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-ovn\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962806    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962832    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-config\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962873    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzvdk\" (UniqueName: \"kubernetes.io/projected/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-kube-api-access-zzvdk\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962929    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ac0c76b-c6f7-44b9-871c-86221abe73a8-konnectivity-ca\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.962976    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.964964 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963007    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cnibin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963045    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-socket-dir-parent\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963096    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-kubelet\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963269    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-systemd-units\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963308    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-slash\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963395    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-systemd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963431    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-script-lib\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963464    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-os-release\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963494    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-kubelet\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963713    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-multus-certs\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29dh7\" (UniqueName: \"kubernetes.io/projected/fbf518f8-7555-4842-8520-a6c58b5aa6f4-kube-api-access-29dh7\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963819    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-log-socket\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963879    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-env-overrides\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963933    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2xn\" (UniqueName: \"kubernetes.io/projected/6364a4d1-df2f-46cd-9f17-335914a957c9-kube-api-access-8r2xn\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963965    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-conf-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.963990    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-daemon-config\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964012    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 11:30:37.965834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964020    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-etc-kubernetes\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964063    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf518f8-7555-4842-8520-a6c58b5aa6f4-host-slash\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964092    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-var-lib-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964122    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-netd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964148    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-netns\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964205    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-multus\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964293    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx9dk\" (UniqueName: \"kubernetes.io/projected/4eb09b6a-2c37-4639-81b8-04b90b536a2a-kube-api-access-wx9dk\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964338    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-node-log\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964380    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964488    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-system-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964521    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964551    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ac0c76b-c6f7-44b9-871c-86221abe73a8-agent-certs\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964619    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c990c3-78d5-4d21-93c9-a474055faa7a-tmp-dir\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964649    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-bin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964678    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/06c990c3-78d5-4d21-93c9-a474055faa7a-hosts-file\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964740    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnr7\" (UniqueName: \"kubernetes.io/projected/06c990c3-78d5-4d21-93c9-a474055faa7a-kube-api-access-2gnr7\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.964788    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cni-binary-copy\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:37.966655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.965739    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:37.969056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.969035    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.970671    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.970792    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-psl4d\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.970803    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.970868    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971033    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971066    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971098    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971279    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971340    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971414    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971592    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pr5s8\""
Jul 28 11:30:37.971799 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.971756    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 11:30:37.973521 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.973506    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:37.975198 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.975033    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xtd8s\""
Jul 28 11:30:37.975683 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.975360    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 11:30:37.975961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.975848    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kc8vp\""
Jul 28 11:30:37.975961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.975855    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 11:30:37.976449 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.976112    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:37.976449 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:37.976172    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:37.976449 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.976304    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 11:30:37.978068 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.978047    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 11:30:37.983983 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:37.983961    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 11:30:38.015730 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.015697    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbzd5"
Jul 28 11:30:38.028385 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.028347    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbzd5"
Jul 28 11:30:38.049131 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.049108    2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 11:30:38.065389 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065346    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbf518f8-7555-4842-8520-a6c58b5aa6f4-iptables-alerter-script\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.065389 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065386    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065410    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-netns\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065433    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovn-node-metrics-cert\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065456    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065480    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-conf-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065507    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-tuned\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065529    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-tmp\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065556    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-ovn\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065581    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.065640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065625    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzvdk\" (UniqueName: \"kubernetes.io/projected/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-kube-api-access-zzvdk\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065645    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cnibin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065664    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-host\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065704    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqbh\" (UniqueName: \"kubernetes.io/projected/398e8ff9-799b-40a1-bd18-c1fd1fe29121-kube-api-access-rkqbh\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-dbus\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-kubelet\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065766    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-systemd-units\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065821    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-slash\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065832    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065845    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-systemd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065911    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-kubelet\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065940    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-socket-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065966    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-sys-fs\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.065990    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6ed79d9-a93c-4e58-9cf8-b270064850fd-host\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066026    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-log-socket\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066049    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-etc-kubernetes\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066074    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-systemd\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066108    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-lib-modules\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066131    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6ed79d9-a93c-4e58-9cf8-b270064850fd-serviceca\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066155    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf518f8-7555-4842-8520-a6c58b5aa6f4-host-slash\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066180    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-netns\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066206    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-kubernetes\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066231    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066256    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c990c3-78d5-4d21-93c9-a474055faa7a-tmp-dir\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066279    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-bin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066304    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-device-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066330    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-var-lib-kubelet\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066371    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-os-release\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/06c990c3-78d5-4d21-93c9-a474055faa7a-hosts-file\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066424    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnr7\" (UniqueName: \"kubernetes.io/projected/06c990c3-78d5-4d21-93c9-a474055faa7a-kube-api-access-2gnr7\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066451    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-k8s-cni-cncf-io\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066463    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbf518f8-7555-4842-8520-a6c58b5aa6f4-iptables-alerter-script\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.066748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066477    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066509    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-etc-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066535    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-bin\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066552    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-conf-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066560    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066590    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-socket-dir-parent\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066614    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066621    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-ovn\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066642    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-config\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066660    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.066668    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ac0c76b-c6f7-44b9-871c-86221abe73a8-konnectivity-ca\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067011    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-bin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067112    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/06c990c3-78d5-4d21-93c9-a474055faa7a-hosts-file\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067260    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c990c3-78d5-4d21-93c9-a474055faa7a-tmp-dir\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067345    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2ac0c76b-c6f7-44b9-871c-86221abe73a8-konnectivity-ca\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067353    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-netns\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067345    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-k8s-cni-cncf-io\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-etc-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.067559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067428    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cnibin\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067455    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-bin\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067505    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-kubelet\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067520    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-socket-dir-parent\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.067577    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.067637    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:38.567616608 +0000 UTC m=+3.024651043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067819    2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067828    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-systemd-units\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067876    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-slash\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067930    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-multus-certs\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.067961    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-conf\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068002    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-run\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068031    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-script-lib\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-os-release\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068057    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-config\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068083    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysconfig\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068107    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.068404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068131    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpgr4\" (UniqueName: \"kubernetes.io/projected/095838fe-7d98-4de4-8909-650ae3f6b8eb-kube-api-access-xpgr4\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068135    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-systemd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068169    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068198    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29dh7\" (UniqueName: \"kubernetes.io/projected/fbf518f8-7555-4842-8520-a6c58b5aa6f4-kube-api-access-29dh7\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068205    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-kubelet\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068223    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-env-overrides\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068253    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2xn\" (UniqueName: \"kubernetes.io/projected/6364a4d1-df2f-46cd-9f17-335914a957c9-kube-api-access-8r2xn\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068108    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-etc-kubernetes\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068474    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-multus-certs\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068608    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-env-overrides\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068832    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-log-socket\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068889    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbf518f8-7555-4842-8520-a6c58b5aa6f4-host-slash\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068926    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovnkube-script-lib\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068965    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-daemon-config\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.068993    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx9dk\" (UniqueName: \"kubernetes.io/projected/4eb09b6a-2c37-4639-81b8-04b90b536a2a-kube-api-access-wx9dk\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069022    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-run-netns\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069020    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-sys\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069077    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-os-release\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.069211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069081    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fm6x\" (UniqueName: \"kubernetes.io/projected/f6ed79d9-a93c-4e58-9cf8-b270064850fd-kube-api-access-4fm6x\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069121    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-var-lib-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069188    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-netd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069225    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-multus\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069290    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-run-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069303    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-registration-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069332    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7lql\" (UniqueName: \"kubernetes.io/projected/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kube-api-access-t7lql\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069355    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cnibin\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069380    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-binary-copy\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069405    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-kubelet-config\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069447    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-node-log\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069538    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-system-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069564    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069588    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-modprobe-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069613    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-system-cni-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.070083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069640    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069669    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ac0c76b-c6f7-44b9-871c-86221abe73a8-agent-certs\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069695    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069722    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cni-binary-copy\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069747    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-hostroot\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069754    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-multus-daemon-config\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069806    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-node-log\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069816    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-hostroot\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069841    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-host-var-lib-cni-multus\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069863    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-run-ovn-kubernetes\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069887    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-var-lib-openvswitch\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069931    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4eb09b6a-2c37-4639-81b8-04b90b536a2a-system-cni-dir\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.069939    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-host-cni-netd\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.070820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.070405    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4eb09b6a-2c37-4639-81b8-04b90b536a2a-cni-binary-copy\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.075663 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.075253    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-ovn-node-metrics-cert\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.075802 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.075689    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2ac0c76b-c6f7-44b9-871c-86221abe73a8-agent-certs\") pod \"konnectivity-agent-st4fs\" (UID: \"2ac0c76b-c6f7-44b9-871c-86221abe73a8\") " pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:38.089077 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.089015    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnr7\" (UniqueName: \"kubernetes.io/projected/06c990c3-78d5-4d21-93c9-a474055faa7a-kube-api-access-2gnr7\") pod \"node-resolver-74fz4\" (UID: \"06c990c3-78d5-4d21-93c9-a474055faa7a\") " pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.093570 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.093514    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:38.093570 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.093544    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:38.093570 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.093558    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:38.093778 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.093622    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:38.593604403 +0000 UTC m=+3.050638842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:38.098775 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.098669    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx9dk\" (UniqueName: \"kubernetes.io/projected/4eb09b6a-2c37-4639-81b8-04b90b536a2a-kube-api-access-wx9dk\") pod \"multus-x929n\" (UID: \"4eb09b6a-2c37-4639-81b8-04b90b536a2a\") " pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.099549 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.099526    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzvdk\" (UniqueName: \"kubernetes.io/projected/dc9375c7-1c2b-4536-92f3-755fcd10a0c0-kube-api-access-zzvdk\") pod \"ovnkube-node-vnt2w\" (UID: \"dc9375c7-1c2b-4536-92f3-755fcd10a0c0\") " pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.100790 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.100765    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2xn\" (UniqueName: \"kubernetes.io/projected/6364a4d1-df2f-46cd-9f17-335914a957c9-kube-api-access-8r2xn\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:38.108784 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.108686    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal" event={"ID":"5742a08682b14dec412b7167019da7bd","Type":"ContainerStarted","Data":"9e353404268e2bd1b69f9b6f4cfceff2e2924a11b7fdc3368cbd0bf6de165082"}
Jul 28 11:30:38.111139 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.111114    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal" event={"ID":"a16892ee32f85011fc1a1806c8666fbf","Type":"ContainerStarted","Data":"e89d96191d158a9252d1e8c508b504b7677bdac040e5a6510c2c40a7d2ee6d85"}
Jul 28 11:30:38.115661 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.115638    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29dh7\" (UniqueName: \"kubernetes.io/projected/fbf518f8-7555-4842-8520-a6c58b5aa6f4-kube-api-access-29dh7\") pod \"iptables-alerter-m8j8l\" (UID: \"fbf518f8-7555-4842-8520-a6c58b5aa6f4\") " pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.170475 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170441    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-tuned\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.170649 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170485    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-tmp\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.170649 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170512    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-host\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.170649 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170537    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqbh\" (UniqueName: \"kubernetes.io/projected/398e8ff9-799b-40a1-bd18-c1fd1fe29121-kube-api-access-rkqbh\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.170649 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170562    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-dbus\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.170851 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170649    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-host\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.170851 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170724    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-dbus\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.170851 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170825    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-socket-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170852    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-sys-fs\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170878    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6ed79d9-a93c-4e58-9cf8-b270064850fd-host\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170933    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-systemd\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170967    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-lib-modules\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.170999    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6ed79d9-a93c-4e58-9cf8-b270064850fd-serviceca\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.171029 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171022    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-kubernetes\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171047    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-device-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171071    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-var-lib-kubelet\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171095    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-os-release\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171097    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6ed79d9-a93c-4e58-9cf8-b270064850fd-host\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171102    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-socket-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171123    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171166    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-systemd\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171148    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-sys-fs\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171214    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-conf\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171226    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171240    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-run\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171270    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysconfig\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171294    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.171316 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171318    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpgr4\" (UniqueName: \"kubernetes.io/projected/095838fe-7d98-4de4-8909-650ae3f6b8eb-kube-api-access-xpgr4\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171345    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171374    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-lib-modules\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171378    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-sys\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171414    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fm6x\" (UniqueName: \"kubernetes.io/projected/f6ed79d9-a93c-4e58-9cf8-b270064850fd-kube-api-access-4fm6x\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171421    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-sys\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171445    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-registration-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171473    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7lql\" (UniqueName: \"kubernetes.io/projected/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kube-api-access-t7lql\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171500    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cnibin\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171528    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-binary-copy\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171539    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-conf\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171555    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-kubelet-config\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171587    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-var-lib-kubelet\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171616    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-modprobe-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171639    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6ed79d9-a93c-4e58-9cf8-b270064850fd-serviceca\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171642    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-system-cni-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171672    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171674    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-system-cni-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171697    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171716    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysconfig\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171719    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-registration-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171731    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171762    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-run\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171272    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-device-dir\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.171864    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-modprobe-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172003    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cnibin\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172057    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172134    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-os-release\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.172212    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.172271    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:30:38.672253631 +0000 UTC m=+3.129288079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172421    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/398e8ff9-799b-40a1-bd18-c1fd1fe29121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172454    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d198ec9-287f-46fe-8097-bdb6554c3c6c-kubelet-config\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172525    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-kubernetes\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.172906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172532    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-sysctl-d\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.173602 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172943    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.173602 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.172992    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/398e8ff9-799b-40a1-bd18-c1fd1fe29121-cni-binary-copy\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.173602 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.173022    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-etc-tuned\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.173602 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.173210    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/095838fe-7d98-4de4-8909-650ae3f6b8eb-tmp\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.201231 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.201201    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqbh\" (UniqueName: \"kubernetes.io/projected/398e8ff9-799b-40a1-bd18-c1fd1fe29121-kube-api-access-rkqbh\") pod \"multus-additional-cni-plugins-zf9gj\" (UID: \"398e8ff9-799b-40a1-bd18-c1fd1fe29121\") " pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.214276 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.214221    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpgr4\" (UniqueName: \"kubernetes.io/projected/095838fe-7d98-4de4-8909-650ae3f6b8eb-kube-api-access-xpgr4\") pod \"tuned-mm7q2\" (UID: \"095838fe-7d98-4de4-8909-650ae3f6b8eb\") " pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.223760 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.223709    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fm6x\" (UniqueName: \"kubernetes.io/projected/f6ed79d9-a93c-4e58-9cf8-b270064850fd-kube-api-access-4fm6x\") pod \"node-ca-r4qzt\" (UID: \"f6ed79d9-a93c-4e58-9cf8-b270064850fd\") " pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.227403 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.227378    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7lql\" (UniqueName: \"kubernetes.io/projected/0fb6253d-435a-4df8-bdae-ad20fc0edaf0-kube-api-access-t7lql\") pod \"aws-ebs-csi-driver-node-w65qj\" (UID: \"0fb6253d-435a-4df8-bdae-ad20fc0edaf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.257665 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.257631    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8j8l"
Jul 28 11:30:38.270988 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.270961    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:30:38.281580 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.281561    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:30:38.287204 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.287185    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-74fz4"
Jul 28 11:30:38.295775 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.295759    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x929n"
Jul 28 11:30:38.304436 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.304417    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj"
Jul 28 11:30:38.314118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.314098    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r4qzt"
Jul 28 11:30:38.321484 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.321463    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm7q2"
Jul 28 11:30:38.328300 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.328276    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zf9gj"
Jul 28 11:30:38.574235 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.574156    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:38.574407 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.574302    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:38.574407 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.574372    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:39.574351048 +0000 UTC m=+4.031385488 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:38.674931 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.674874    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:38.675107 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:38.675001    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:38.675107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675052    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:38.675107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675074    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:38.675107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675083    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:38.675107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675103    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:38.675306 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675135    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:39.675120434 +0000 UTC m=+4.132154869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:38.675306 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:38.675157    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:30:39.675141789 +0000 UTC m=+4.132176247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:39.031595 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.031512    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 11:25:38 +0000 UTC" deadline="2028-04-29 16:25:45.843792902 +0000 UTC"
Jul 28 11:30:39.031595 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.031546    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15388h55m6.812251959s"
Jul 28 11:30:39.237105 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:39.237073    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398e8ff9_799b_40a1_bd18_c1fd1fe29121.slice/crio-fe53229b3640ca49fadbdd41becdd043c21ae14c48607b7a2faf2600a9c0578f WatchSource:0}: Error finding container fe53229b3640ca49fadbdd41becdd043c21ae14c48607b7a2faf2600a9c0578f: Status 404 returned error can't find the container with id fe53229b3640ca49fadbdd41becdd043c21ae14c48607b7a2faf2600a9c0578f
Jul 28 11:30:39.243191 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:39.243160    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb6253d_435a_4df8_bdae_ad20fc0edaf0.slice/crio-f90f452040625133ea82baf5ff8191811313c10f2f82a3c56a3d65871bec0b52 WatchSource:0}: Error finding container f90f452040625133ea82baf5ff8191811313c10f2f82a3c56a3d65871bec0b52: Status 404 returned error can't find the container with id f90f452040625133ea82baf5ff8191811313c10f2f82a3c56a3d65871bec0b52
Jul 28 11:30:39.243615 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:39.243480    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9375c7_1c2b_4536_92f3_755fcd10a0c0.slice/crio-77831fdea080bd560cbf45379d4f287bba03fc9d6d2288536e10459c5c6f668c WatchSource:0}: Error finding container 77831fdea080bd560cbf45379d4f287bba03fc9d6d2288536e10459c5c6f668c: Status 404 returned error can't find the container with id 77831fdea080bd560cbf45379d4f287bba03fc9d6d2288536e10459c5c6f668c
Jul 28 11:30:39.243859 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:30:39.243837    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb09b6a_2c37_4639_81b8_04b90b536a2a.slice/crio-05787cbfe5b22b8960f199e2ac955baab0645584c66545bbee0acfc7f941bfa7 WatchSource:0}: Error finding container 05787cbfe5b22b8960f199e2ac955baab0645584c66545bbee0acfc7f941bfa7: Status 404 returned error can't find the container with id 05787cbfe5b22b8960f199e2ac955baab0645584c66545bbee0acfc7f941bfa7
Jul 28 11:30:39.572426 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.572193    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 11:30:39.582233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.582204    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:39.582363 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.582345    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:39.582415 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.582406    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:41.582391394 +0000 UTC m=+6.039425829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:39.682932 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.682888    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:39.682932 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:39.682941    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683057    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683076    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683090    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683099    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683127    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:30:41.683107231 +0000 UTC m=+6.140141667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:39.683152 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:39.683146    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:41.683136938 +0000 UTC m=+6.140171378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:40.032403 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.032209    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 11:25:38 +0000 UTC" deadline="2028-02-26 02:40:35.205864267 +0000 UTC"
Jul 28 11:30:40.032403 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.032271    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13863h9m55.173597667s"
Jul 28 11:30:40.102933 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.102811    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:40.102933 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.102867    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:40.102933 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.102881    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:40.103235 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:40.103022    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:40.103442 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:40.103416    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:40.103531 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:40.103511    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:40.130913 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.130122    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal" event={"ID":"a16892ee32f85011fc1a1806c8666fbf","Type":"ContainerStarted","Data":"cc5228e9a36a449b52dc0262c3880a72def8066521c59574ce59408bfd63d994"}
Jul 28 11:30:40.139840 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.139743    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8j8l" event={"ID":"fbf518f8-7555-4842-8520-a6c58b5aa6f4","Type":"ContainerStarted","Data":"6ffcbff592ce6c8fb8beb370db6a9df8df026cb4464412d61220a6bdf4e3bf34"}
Jul 28 11:30:40.144293 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.144264    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r4qzt" event={"ID":"f6ed79d9-a93c-4e58-9cf8-b270064850fd","Type":"ContainerStarted","Data":"eadfc423367e225614fe552a06c3935713ec1722ee153f8afb7834289b0c0733"}
Jul 28 11:30:40.156220 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.156185    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-74fz4" event={"ID":"06c990c3-78d5-4d21-93c9-a474055faa7a","Type":"ContainerStarted","Data":"82c088cbb4a5260f567533edd9e110005c7e98d96c402fb4086567b32c3d4084"}
Jul 28 11:30:40.162671 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.162642    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"77831fdea080bd560cbf45379d4f287bba03fc9d6d2288536e10459c5c6f668c"}
Jul 28 11:30:40.170610 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.170578    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm7q2" event={"ID":"095838fe-7d98-4de4-8909-650ae3f6b8eb","Type":"ContainerStarted","Data":"bdff3ffad02d5ee0c33d1e5730891baeb6ccb881495faa8e7a1f9fc15ccfc11d"}
Jul 28 11:30:40.182361 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.181003    2552 generic.go:356] "Generic (PLEG): container finished" podID="5742a08682b14dec412b7167019da7bd" containerID="e9a0ad8b6f2ed3591a2d45990ff44d15c4b5ffae72ee83c66cd29081f114d1ec" exitCode=0
Jul 28 11:30:40.182361 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.181110    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal" event={"ID":"5742a08682b14dec412b7167019da7bd","Type":"ContainerDied","Data":"e9a0ad8b6f2ed3591a2d45990ff44d15c4b5ffae72ee83c66cd29081f114d1ec"}
Jul 28 11:30:40.195823 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.195753    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-st4fs" event={"ID":"2ac0c76b-c6f7-44b9-871c-86221abe73a8","Type":"ContainerStarted","Data":"16613ef1d9e990ade5a8087720c1499444325be6624d2b61d76945e06778abe1"}
Jul 28 11:30:40.199151 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.199100    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x929n" event={"ID":"4eb09b6a-2c37-4639-81b8-04b90b536a2a","Type":"ContainerStarted","Data":"05787cbfe5b22b8960f199e2ac955baab0645584c66545bbee0acfc7f941bfa7"}
Jul 28 11:30:40.203829 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.202626    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-140.ec2.internal" podStartSLOduration=3.202608459 podStartE2EDuration="3.202608459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:40.15180084 +0000 UTC m=+4.608835308" watchObservedRunningTime="2026-07-28 11:30:40.202608459 +0000 UTC m=+4.659642917"
Jul 28 11:30:40.210844 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.210819    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj" event={"ID":"0fb6253d-435a-4df8-bdae-ad20fc0edaf0","Type":"ContainerStarted","Data":"f90f452040625133ea82baf5ff8191811313c10f2f82a3c56a3d65871bec0b52"}
Jul 28 11:30:40.215480 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:40.215417    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerStarted","Data":"fe53229b3640ca49fadbdd41becdd043c21ae14c48607b7a2faf2600a9c0578f"}
Jul 28 11:30:41.235068 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:41.234986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal" event={"ID":"5742a08682b14dec412b7167019da7bd","Type":"ContainerStarted","Data":"d2b5857c8c1f890d19565c7aca2f4a64d30ca363c906f7de278239568425ef41"}
Jul 28 11:30:41.602053 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:41.601976    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:41.602195 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.602120    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:41.602195 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.602181    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:45.602163197 +0000 UTC m=+10.059197638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:41.703297 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:41.703172    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:41.703297 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:41.703226    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703341    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703361    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703384    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703399    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703412    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:30:45.703392934 +0000 UTC m=+10.160427372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:41.703539 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:41.703448    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:45.703433201 +0000 UTC m=+10.160467640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:42.103111 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:42.103075    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:42.103281 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:42.103110    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:42.103281 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:42.103237    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:42.103701 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:42.103673    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:42.104040 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:42.103879    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:42.104040 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:42.103988    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:44.104289 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:44.104254    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:44.104739 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:44.104395    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:44.104739 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:44.104636    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:44.104858 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:44.104737    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:44.104858 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:44.104800    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:44.104988 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:44.104871    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:45.638981 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:45.638949    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:45.639445 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.639090    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:45.639445 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.639138    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:53.639124215 +0000 UTC m=+18.096158650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:45.739559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:45.739511    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:45.739739 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:45.739565    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:45.739739 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.739733    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:45.739849 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.739752    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:45.739849 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.739767    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:45.739849 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.739831    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:30:53.739810842 +0000 UTC m=+18.196845279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:45.740028 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.739971    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:45.740028 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:45.740026    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:30:53.74001265 +0000 UTC m=+18.197047101 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:46.103849 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:46.103720    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:46.103849 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:46.103826    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:46.104090 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:46.103905    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:46.104090 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:46.104018    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:46.104090 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:46.104063    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:46.104239 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:46.104142    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:48.102740 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:48.102702    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:48.103226 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:48.102817    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:48.103226 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:48.102703    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:48.103226 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:48.102875    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:48.103226 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:48.103010    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:48.103226 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:48.103134    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:50.103363 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:50.103284    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:50.103363 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:50.103339    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:50.103802 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:50.103370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:50.103802 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:50.103479    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:50.103802 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:50.103539    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:50.103802 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:50.103636    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:52.102788 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:52.102756    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:52.103243 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:52.102758    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:52.103243 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:52.102872    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:52.103243 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:52.102890    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:52.103243 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:52.102973    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:52.103243 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:52.103052    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:53.700114 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:53.700072    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:53.700655 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.700226    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:53.700655 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.700305    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:31:09.700283004 +0000 UTC m=+34.157317459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:30:53.801056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:53.801019    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:53.801200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:53.801114    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:53.801272 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801206    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:53.801272 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801208    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:30:53.801272 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801239    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:30:53.801272 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801253    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:53.801444 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801255    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:09.80124157 +0000 UTC m=+34.258276005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:30:53.801444 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:53.801305    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:31:09.801293681 +0000 UTC m=+34.258328116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:30:54.102635 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:54.102561    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:54.102791 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:54.102561    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:54.102791 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:54.102679    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:54.102791 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:54.102570    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:54.102791 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:54.102756    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:54.102987 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:54.102822    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:56.103558 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:56.103459    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:56.103558 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:56.103497    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:56.104056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:56.103620    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:56.104056 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:56.103629    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:56.104056 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:56.103736    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:56.104056 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:56.103817    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:58.102668 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.102405    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:30:58.103499 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.102410    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:30:58.103499 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:58.102745    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:30:58.103499 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.102471    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:30:58.103499 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:58.102829    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:30:58.103499 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:30:58.102889    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:30:58.269637 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.269556    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r4qzt" event={"ID":"f6ed79d9-a93c-4e58-9cf8-b270064850fd","Type":"ContainerStarted","Data":"8a7be919c005261d321870912da8d46f122aa1375a462683f14767c7ad8b8f91"}
Jul 28 11:30:58.270972 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.270950    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-74fz4" event={"ID":"06c990c3-78d5-4d21-93c9-a474055faa7a","Type":"ContainerStarted","Data":"551edeb21860bc3448f2b5cfded49936e8940517eeb545f3a806994b7ecf6844"}
Jul 28 11:30:58.272845 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.272827    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"f64fd5687790b89dcd1b651d33fdc2615f8f44801c938497381a846f3dcb4ec9"}
Jul 28 11:30:58.272921 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.272885    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"ebd8b9d2d7d0cad9f9b82999fc5d5b74a81a63065bc7684e68ae84568ed5be7a"}
Jul 28 11:30:58.273020 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.272935    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"64db08b69282d504db03e01b94569268f7f677ce4fd131cfe23ce26eba067731"}
Jul 28 11:30:58.273020 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.272972    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"2fcaf559a1757bce30573b985f7b6ad39cff9f94d9274a1bd8668ed3fe25d75e"}
Jul 28 11:30:58.274226 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.274207    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm7q2" event={"ID":"095838fe-7d98-4de4-8909-650ae3f6b8eb","Type":"ContainerStarted","Data":"36456c2465ecd03ca9678cd61e2a185ebf7f314eb044f6fc0514e4e1325c11dd"}
Jul 28 11:30:58.275530 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.275509    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-st4fs" event={"ID":"2ac0c76b-c6f7-44b9-871c-86221abe73a8","Type":"ContainerStarted","Data":"dc79ac00839c0d756ae5c9f383a57d45ac7891e8124ff45718fc0e0d4251cabf"}
Jul 28 11:30:58.276835 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.276816    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x929n" event={"ID":"4eb09b6a-2c37-4639-81b8-04b90b536a2a","Type":"ContainerStarted","Data":"1a981c1710fc6c0a902d5358346f733fdf0c860658165b6316262501f780cb0a"}
Jul 28 11:30:58.278033 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.278010    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj" event={"ID":"0fb6253d-435a-4df8-bdae-ad20fc0edaf0","Type":"ContainerStarted","Data":"b8ab0b3cc5536b1cb092df30d1d46f7449a1b5dad8f36457864d6c1e9cad0410"}
Jul 28 11:30:58.279371 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.279351    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="aa857b92db5a04cc1b3ec7cb674eb3fa21eab24f1a5a2888e66ded1480085054" exitCode=0
Jul 28 11:30:58.279433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.279384    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"aa857b92db5a04cc1b3ec7cb674eb3fa21eab24f1a5a2888e66ded1480085054"}
Jul 28 11:30:58.284393 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.284353    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-140.ec2.internal" podStartSLOduration=21.284342101 podStartE2EDuration="21.284342101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:41.254759652 +0000 UTC m=+5.711794101" watchObservedRunningTime="2026-07-28 11:30:58.284342101 +0000 UTC m=+22.741376558"
Jul 28 11:30:58.284995 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.284962    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r4qzt" podStartSLOduration=11.743937929 podStartE2EDuration="22.284949387s" totalImagesPullingTime="10.541011458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:58.284507772 +0000 UTC m=+22.741542247" watchObservedRunningTime="2026-07-28 11:30:58.284949387 +0000 UTC m=+22.741983845"
Jul 28 11:30:58.300247 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.300209    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x929n" podStartSLOduration=3.692751377 podStartE2EDuration="22.300199246s" totalImagesPullingTime="18.607447869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:58.299777668 +0000 UTC m=+22.756812125" watchObservedRunningTime="2026-07-28 11:30:58.300199246 +0000 UTC m=+22.757233702"
Jul 28 11:30:58.322401 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.322366    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-st4fs" podStartSLOduration=3.781746016 podStartE2EDuration="22.322355477s" totalImagesPullingTime="18.540609461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:58.322196592 +0000 UTC m=+22.779231070" watchObservedRunningTime="2026-07-28 11:30:58.322355477 +0000 UTC m=+22.779389933"
Jul 28 11:30:58.338863 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.338828    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-74fz4" podStartSLOduration=3.799427824 podStartE2EDuration="22.338816745s" totalImagesPullingTime="18.539388921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:58.338450389 +0000 UTC m=+22.795484846" watchObservedRunningTime="2026-07-28 11:30:58.338816745 +0000 UTC m=+22.795851201"
Jul 28 11:30:58.389795 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:58.388148    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mm7q2" podStartSLOduration=3.8382522 podStartE2EDuration="22.388127112s" totalImagesPullingTime="18.549874912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:58.357273727 +0000 UTC m=+22.814308184" watchObservedRunningTime="2026-07-28 11:30:58.388127112 +0000 UTC m=+22.845161569"
Jul 28 11:30:59.217080 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.217042    2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 11:30:59.283707 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.283671    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj" event={"ID":"0fb6253d-435a-4df8-bdae-ad20fc0edaf0","Type":"ContainerStarted","Data":"46a2f27f4ebc92a07b600e90fa98398740712829f05cf9b430addf5ad3b8fe1f"}
Jul 28 11:30:59.285395 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.285366    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8j8l" event={"ID":"fbf518f8-7555-4842-8520-a6c58b5aa6f4","Type":"ContainerStarted","Data":"2f066c184ffab37161e18771ba83f71c1e2373f73d6b70bd38f075efc828efff"}
Jul 28 11:30:59.288660 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.288635    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"aa1e603d5d3a838ad0c15aed65b0bb725344164ffc55884e3cea9c0300d217ea"}
Jul 28 11:30:59.288770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.288718    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"31806f48cfd10a3199532eb64854a543f07a82badd23b97e063660886a01667a"}
Jul 28 11:30:59.307435 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:30:59.307389    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m8j8l" podStartSLOduration=4.767810143 podStartE2EDuration="23.307375314s" totalImagesPullingTime="18.539565171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:30:59.307145016 +0000 UTC m=+23.764179476" watchObservedRunningTime="2026-07-28 11:30:59.307375314 +0000 UTC m=+23.764409770"
Jul 28 11:31:00.027126 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.027010    2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T11:30:59.217063107Z","UUID":"f08bbbef-2017-4ade-8085-0e7517979817","Handler":null,"Name":"","Endpoint":""}
Jul 28 11:31:00.028590 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.028441    2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 11:31:00.028590 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.028493    2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 11:31:00.103253 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.103214    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:00.103426 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.103220    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:00.103426 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:00.103352    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:00.103426 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:00.103370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:00.103595 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:00.103461    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:00.103595 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:00.103550    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:01.295866 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:01.295630    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj" event={"ID":"0fb6253d-435a-4df8-bdae-ad20fc0edaf0","Type":"ContainerStarted","Data":"fed443c9cc68bcebd9f7c57b5822e6c40505ed8fa933e9924a58dbdd7c7d864a"}
Jul 28 11:31:01.299478 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:01.299444    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"58388d4a40432d9b44ff8c353f9ef04fb1f049e0193570f4f9797b6c9e4bd6fe"}
Jul 28 11:31:01.319177 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:01.319134    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w65qj" podStartSLOduration=4.583704225 podStartE2EDuration="25.319116039s" totalImagesPullingTime="20.735411814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:01.318648749 +0000 UTC m=+25.775683219" watchObservedRunningTime="2026-07-28 11:31:01.319116039 +0000 UTC m=+25.776150528"
Jul 28 11:31:02.019573 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:02.019539    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:31:02.020407 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:02.020381    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:31:02.102838 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:02.102800    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:02.103036 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:02.102860    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:02.103036 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:02.102870    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:02.103036 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:02.102985    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:02.103206 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:02.103108    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:02.103255 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:02.103198    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:03.308554 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:03.308339    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" event={"ID":"dc9375c7-1c2b-4536-92f3-755fcd10a0c0","Type":"ContainerStarted","Data":"fe9dba26c1ae0b47c23f3b667141a81d92e014eef9ba6089d87aca346a0b24c5"}
Jul 28 11:31:03.310502 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:03.308643    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:03.324035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:03.324015    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:03.344821 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:03.344778    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w" podStartSLOduration=8.732496786 podStartE2EDuration="27.344761974s" totalImagesPullingTime="18.612265188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:03.344315318 +0000 UTC m=+27.801349774" watchObservedRunningTime="2026-07-28 11:31:03.344761974 +0000 UTC m=+27.801796431"
Jul 28 11:31:04.103625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.103457    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:04.103832 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.103455    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:04.103832 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.103500    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:04.103832 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:04.103814    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:04.103992 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:04.103870    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:04.103992 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:04.103700    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:04.312176 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.312145    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="f4e430ab6713c4c6f31a0943778fce0f85537a78354ceffcf32d1bd7ee001d77" exitCode=0
Jul 28 11:31:04.312637 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.312239    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"f4e430ab6713c4c6f31a0943778fce0f85537a78354ceffcf32d1bd7ee001d77"}
Jul 28 11:31:04.312700 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.312645    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:04.312748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.312718    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:04.327601 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:04.327575    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:05.290352 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.290327    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:31:05.290868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.290834    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-st4fs"
Jul 28 11:31:05.641796 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.641764    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rsxxg"]
Jul 28 11:31:05.642132 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.641866    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:05.642132 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:05.641968    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:05.645830 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.645806    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dqwqh"]
Jul 28 11:31:05.645959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.645910    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:05.646016 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:05.645987    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:05.659233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.658707    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2dpkp"]
Jul 28 11:31:05.659233 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:05.658867    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:05.659233 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:05.659002    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:06.319002 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:06.318828    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="a2fcf45c82898d498fac0c3e45dff39fff109962655ed7db1429563211c4139b" exitCode=0
Jul 28 11:31:06.319002 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:06.318943    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"a2fcf45c82898d498fac0c3e45dff39fff109962655ed7db1429563211c4139b"}
Jul 28 11:31:07.102640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:07.102609    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:07.103072 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:07.102711    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:07.103072 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:07.102717    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:07.103072 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:07.102747    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:07.103072 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:07.102850    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:07.103072 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:07.102973    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:08.324925 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:08.324874    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="21fa8ba08bb070a3e359d609982367e436b3578f7a9544c94a0b8798d1d11a27" exitCode=0
Jul 28 11:31:08.325431 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:08.324927    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"21fa8ba08bb070a3e359d609982367e436b3578f7a9544c94a0b8798d1d11a27"}
Jul 28 11:31:09.103513 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.103478    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:09.103692 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.103491    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:09.103692 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.103614    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dpkp" podUID="8f270c87-7947-4f93-af8f-b078821f49f1"
Jul 28 11:31:09.103692 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.103625    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:09.103934 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.103705    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:31:09.103934 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.103793    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rsxxg" podUID="2d198ec9-287f-46fe-8097-bdb6554c3c6c"
Jul 28 11:31:09.720855 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.720807    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:09.721376 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.720944    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:31:09.721376 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.721021    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:31:41.721004117 +0000 UTC m=+66.178038556 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 11:31:09.822263 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.822231    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:09.822263 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:09.822266    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:09.822486 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822389    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 11:31:09.822486 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822458    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret podName:2d198ec9-287f-46fe-8097-bdb6554c3c6c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:41.822436773 +0000 UTC m=+66.279471234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret") pod "global-pull-secret-syncer-rsxxg" (UID: "2d198ec9-287f-46fe-8097-bdb6554c3c6c") : object "kube-system"/"original-pull-secret" not registered
Jul 28 11:31:09.822583 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822394    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 11:31:09.822583 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822510    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 11:31:09.822583 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822525    2552 projected.go:196] Error preparing data for projected volume kube-api-access-qzx7h for pod openshift-network-diagnostics/network-check-target-2dpkp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:31:09.822583 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:09.822572    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h podName:8f270c87-7947-4f93-af8f-b078821f49f1 nodeName:}" failed. No retries permitted until 2026-07-28 11:31:41.822557551 +0000 UTC m=+66.279591990 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzx7h" (UniqueName: "kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h") pod "network-check-target-2dpkp" (UID: "8f270c87-7947-4f93-af8f-b078821f49f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 11:31:10.353445 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.353159    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-140.ec2.internal" event="NodeReady"
Jul 28 11:31:10.353445 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.353326    2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 11:31:10.492793 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.492759    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:31:10.496878 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.496858    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.503247 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.503221    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 11:31:10.504284 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.503483    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 11:31:10.504284 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.503526    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxmgd\""
Jul 28 11:31:10.507123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.506975    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 11:31:10.515175 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.515158    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 11:31:10.528294 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.528164    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:31:10.546390 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.546363    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"]
Jul 28 11:31:10.551750 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.551732    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"]
Jul 28 11:31:10.551907 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.551877    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:10.555366 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.555350    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"
Jul 28 11:31:10.555979 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.555953    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-wxnqj\" is forbidden: User \"system:node:ip-10-0-130-140.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-130-140.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wxnqj\"" type="*v1.Secret"
Jul 28 11:31:10.557296 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.557262    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-130-140.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-130-140.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap"
Jul 28 11:31:10.557296 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.557283    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-130-140.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-130-140.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret"
Jul 28 11:31:10.560712 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.560691    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 11:31:10.560994 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.560964    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fkxzj\""
Jul 28 11:31:10.561087 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.560967    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 11:31:10.593099 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.593073    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"]
Jul 28 11:31:10.593959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.593931    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"]
Jul 28 11:31:10.600002 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.599972    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9jzkc"]
Jul 28 11:31:10.603909 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.603878    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rpzw5"]
Jul 28 11:31:10.604035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.604011    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.606058 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.606038    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l5ffw\""
Jul 28 11:31:10.606058 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.606062    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 11:31:10.606515 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.606497    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 11:31:10.607559 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.607537    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.614983 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.614964    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 11:31:10.615136 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.615115    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 11:31:10.615218 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.615159    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 11:31:10.615218 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.615187    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zjgd5\""
Jul 28 11:31:10.628842 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.628820    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.628959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.628847    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.628959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.628871    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.628959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.628912    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.629104 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.629015    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.629104 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.629072    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.629199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.629122    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.629199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.629184    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4pmh\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.638031 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.637997    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rpzw5"]
Jul 28 11:31:10.638693 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.638661    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9jzkc"]
Jul 28 11:31:10.729792 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729757    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.729792 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729796    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9xjx\" (UniqueName: \"kubernetes.io/projected/f70ee650-e643-4722-bf6d-cfac90f5859c-kube-api-access-x9xjx\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729830    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729853    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729886    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729927    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwbrw\" (UniqueName: \"kubernetes.io/projected/2523e3d8-921f-4011-90b3-810e330a73aa-kube-api-access-bwbrw\") pod \"network-check-source-7d88cb55d8-9lgq9\" (UID: \"2523e3d8-921f-4011-90b3-810e330a73aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729953    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e625692f-e4a7-46c7-b432-ef63006556ed-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.729981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730024    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730059    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730085    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70ee650-e643-4722-bf6d-cfac90f5859c-tmp-dir\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.730230    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.730249    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:10.730400 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.730318    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:11.230297245 +0000 UTC m=+35.687331699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730545    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730598    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f70ee650-e643-4722-bf6d-cfac90f5859c-config-volume\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730629    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a63620b-e497-402b-9ea0-864e8f1df157-cert\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730654    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgpz2\" (UniqueName: \"kubernetes.io/projected/8a63620b-e497-402b-9ea0-864e8f1df157-kube-api-access-sgpz2\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730686    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4pmh\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730724    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.730964    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.731088 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.731088    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.731428 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.731328    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.734873 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.734855    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.734873 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.734865    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.760520 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.760492    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4pmh\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.763087 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.763064    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:10.831376 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831289    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f70ee650-e643-4722-bf6d-cfac90f5859c-config-volume\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.831376 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831337    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a63620b-e497-402b-9ea0-864e8f1df157-cert\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.831376 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831363    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgpz2\" (UniqueName: \"kubernetes.io/projected/8a63620b-e497-402b-9ea0-864e8f1df157-kube-api-access-sgpz2\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.831640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831390    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:10.831640 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831424    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.831640 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.831491    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:10.831640 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:10.831550    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:11.331531996 +0000 UTC m=+35.788566432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:10.831847 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831656    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9xjx\" (UniqueName: \"kubernetes.io/projected/f70ee650-e643-4722-bf6d-cfac90f5859c-kube-api-access-x9xjx\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.831847 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbrw\" (UniqueName: \"kubernetes.io/projected/2523e3d8-921f-4011-90b3-810e330a73aa-kube-api-access-bwbrw\") pod \"network-check-source-7d88cb55d8-9lgq9\" (UID: \"2523e3d8-921f-4011-90b3-810e330a73aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"
Jul 28 11:31:10.831847 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831835    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e625692f-e4a7-46c7-b432-ef63006556ed-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:10.832027 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.831933    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70ee650-e643-4722-bf6d-cfac90f5859c-tmp-dir\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.832027 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.832000    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f70ee650-e643-4722-bf6d-cfac90f5859c-config-volume\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.832178 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.832163    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70ee650-e643-4722-bf6d-cfac90f5859c-tmp-dir\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.834215 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.834187    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a63620b-e497-402b-9ea0-864e8f1df157-cert\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.850393 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.850364    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwbrw\" (UniqueName: \"kubernetes.io/projected/2523e3d8-921f-4011-90b3-810e330a73aa-kube-api-access-bwbrw\") pod \"network-check-source-7d88cb55d8-9lgq9\" (UID: \"2523e3d8-921f-4011-90b3-810e330a73aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"
Jul 28 11:31:10.850992 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.850968    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgpz2\" (UniqueName: \"kubernetes.io/projected/8a63620b-e497-402b-9ea0-864e8f1df157-kube-api-access-sgpz2\") pod \"ingress-canary-rpzw5\" (UID: \"8a63620b-e497-402b-9ea0-864e8f1df157\") " pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:10.851135 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.851119    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9xjx\" (UniqueName: \"kubernetes.io/projected/f70ee650-e643-4722-bf6d-cfac90f5859c-kube-api-access-x9xjx\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:10.868700 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.868674    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"
Jul 28 11:31:10.920762 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:10.920734    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rpzw5"
Jul 28 11:31:11.102512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.102427    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:11.102663 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.102554    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:11.102663 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.102584    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:11.112277 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.112251    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2mvc2\""
Jul 28 11:31:11.113114 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.113093    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 11:31:11.113218 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.113093    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tk4rs\""
Jul 28 11:31:11.113627 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.113605    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 11:31:11.235176 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.235146    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:11.235343 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.235323    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:11.235389 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.235345    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:11.235426 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.235419    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:12.235400101 +0000 UTC m=+36.692434541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:11.336386 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.336350    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:11.336575 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.336515    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:11.336646 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.336585    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:12.336565868 +0000 UTC m=+36.793600303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:11.605102 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.605062    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 11:31:11.611650 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.611622    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:11.611786 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:11.611698    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:12.111678194 +0000 UTC m=+36.568712629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:11.712142 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.712111    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 11:31:11.722737 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.722703    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e625692f-e4a7-46c7-b432-ef63006556ed-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:11.940871 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:11.940792    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wxnqj\""
Jul 28 11:31:12.143670 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:12.143627    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:12.143863 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.143830    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:12.143972 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.143930    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:13.143907731 +0000 UTC m=+37.600942185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:12.244817 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:12.244777    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:12.245009 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.244962    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:12.245009 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.244985    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:12.245107 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.245091    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:14.245067898 +0000 UTC m=+38.702102337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:12.347109 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:12.345543    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:12.347109 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.346419    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:12.347109 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:12.346498    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:14.346477047 +0000 UTC m=+38.803511485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:12.439682 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:12.439646    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rpzw5"]
Jul 28 11:31:12.440585 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:12.440562    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9"]
Jul 28 11:31:12.442730 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:31:12.442704    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a63620b_e497_402b_9ea0_864e8f1df157.slice/crio-36184f074a50e5a8ae3bc8b9677afc9b6860bd61d1ea6f459d71881fec5bd9d9 WatchSource:0}: Error finding container 36184f074a50e5a8ae3bc8b9677afc9b6860bd61d1ea6f459d71881fec5bd9d9: Status 404 returned error can't find the container with id 36184f074a50e5a8ae3bc8b9677afc9b6860bd61d1ea6f459d71881fec5bd9d9
Jul 28 11:31:12.443327 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:31:12.443291    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2523e3d8_921f_4011_90b3_810e330a73aa.slice/crio-0d72d8f37f1865f0dbd57ec2df909e057f8919c69bb6a2f55878f7497afb0485 WatchSource:0}: Error finding container 0d72d8f37f1865f0dbd57ec2df909e057f8919c69bb6a2f55878f7497afb0485: Status 404 returned error can't find the container with id 0d72d8f37f1865f0dbd57ec2df909e057f8919c69bb6a2f55878f7497afb0485
Jul 28 11:31:13.151469 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:13.151292    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:13.151469 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:13.151419    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:13.152032 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:13.151487    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:15.151468154 +0000 UTC m=+39.608502588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:13.340747 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:13.340706    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rpzw5" event={"ID":"8a63620b-e497-402b-9ea0-864e8f1df157","Type":"ContainerStarted","Data":"36184f074a50e5a8ae3bc8b9677afc9b6860bd61d1ea6f459d71881fec5bd9d9"}
Jul 28 11:31:13.346389 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:13.346292    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="296c30c8e8731ecab38012e703cbba90b411cf2ea60f4377a786d606ae17f056" exitCode=0
Jul 28 11:31:13.346389 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:13.346345    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"296c30c8e8731ecab38012e703cbba90b411cf2ea60f4377a786d606ae17f056"}
Jul 28 11:31:13.348599 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:13.347843    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9" event={"ID":"2523e3d8-921f-4011-90b3-810e330a73aa","Type":"ContainerStarted","Data":"0d72d8f37f1865f0dbd57ec2df909e057f8919c69bb6a2f55878f7497afb0485"}
Jul 28 11:31:14.260655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:14.260618    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:14.261129 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:14.260766    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:14.261129 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:14.260791    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:14.261129 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:14.260848    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:18.260833898 +0000 UTC m=+42.717868338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:14.355681 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:14.355645    2552 generic.go:356] "Generic (PLEG): container finished" podID="398e8ff9-799b-40a1-bd18-c1fd1fe29121" containerID="d62b97a77ddcd54e7d639ff7ff69ed4b09ff21d430005d4d43b55525a351fdd4" exitCode=0
Jul 28 11:31:14.355845 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:14.355709    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerDied","Data":"d62b97a77ddcd54e7d639ff7ff69ed4b09ff21d430005d4d43b55525a351fdd4"}
Jul 28 11:31:14.361782 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:14.361755    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:14.361909 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:14.361872    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:14.361985 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:14.361949    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:18.361930048 +0000 UTC m=+42.818964496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:15.169536 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:15.169492    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:15.169725 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:15.169670    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:15.169803 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:15.169757    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:19.169729024 +0000 UTC m=+43.626763463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:17.365062 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:17.364954    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9" event={"ID":"2523e3d8-921f-4011-90b3-810e330a73aa","Type":"ContainerStarted","Data":"df24d7b42f9138a0fc0ec0b58b425c2b1cab5bcdc98d3e276e824b8314a43862"}
Jul 28 11:31:17.367075 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:17.367040    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rpzw5" event={"ID":"8a63620b-e497-402b-9ea0-864e8f1df157","Type":"ContainerStarted","Data":"fd444d53b5f0d65b73b6455a96138ab633aad388df1d52448d39fe800b7b2d3b"}
Jul 28 11:31:17.370432 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:17.370410    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" event={"ID":"398e8ff9-799b-40a1-bd18-c1fd1fe29121","Type":"ContainerStarted","Data":"b799154b8be875e4732d7267f4a265c8f1e2ef84141a3017232bf615c6b4080e"}
Jul 28 11:31:17.394912 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:17.394850    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9lgq9" podStartSLOduration=32.387845348 podStartE2EDuration="36.394832203s" totalImagesPullingTime="4.006986855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:17.394427361 +0000 UTC m=+41.851461815" watchObservedRunningTime="2026-07-28 11:31:17.394832203 +0000 UTC m=+41.851866663"
Jul 28 11:31:17.451706 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:17.451664    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zf9gj" podStartSLOduration=11.753411307 podStartE2EDuration="41.451651073s" totalImagesPullingTime="29.698239766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:17.436959957 +0000 UTC m=+41.893994414" watchObservedRunningTime="2026-07-28 11:31:17.451651073 +0000 UTC m=+41.908685530"
Jul 28 11:31:18.290526 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:18.290483    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:18.290704 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:18.290626    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:18.290704 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:18.290646    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:18.290704 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:18.290699    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:26.290685781 +0000 UTC m=+50.747720216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:18.391844 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:18.391810    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:18.392301 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:18.391981    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:18.392301 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:18.392098    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:26.392072813 +0000 UTC m=+50.849107260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:19.197083 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:19.197041    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:19.197234 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:19.197180    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:19.197297 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:19.197267    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:27.197249866 +0000 UTC m=+51.654284313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:26.072811 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.072760    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rpzw5" podStartSLOduration=12.073016675 podStartE2EDuration="16.072745381s" totalImagesPullingTime="3.999728706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:17.490123541 +0000 UTC m=+41.947157998" watchObservedRunningTime="2026-07-28 11:31:26.072745381 +0000 UTC m=+50.529779838"
Jul 28 11:31:26.073518 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.073501    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"]
Jul 28 11:31:26.119995 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.119961    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.123309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.123287    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"]
Jul 28 11:31:26.135986 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.135963    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 11:31:26.136378 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.136343    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 11:31:26.136625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.136605    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 11:31:26.137993 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.137974    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 11:31:26.149886 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.149866    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/46cda2cb-ea01-482c-a34d-c27181a010c4-klusterlet-config\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.149987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.149930    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ch8w\" (UniqueName: \"kubernetes.io/projected/46cda2cb-ea01-482c-a34d-c27181a010c4-kube-api-access-9ch8w\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.150037 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.149991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46cda2cb-ea01-482c-a34d-c27181a010c4-tmp\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.250320 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.250288    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/46cda2cb-ea01-482c-a34d-c27181a010c4-klusterlet-config\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.250471 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.250344    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ch8w\" (UniqueName: \"kubernetes.io/projected/46cda2cb-ea01-482c-a34d-c27181a010c4-kube-api-access-9ch8w\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.250471 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.250381    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46cda2cb-ea01-482c-a34d-c27181a010c4-tmp\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.250791 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.250769    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46cda2cb-ea01-482c-a34d-c27181a010c4-tmp\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.254337 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.254306    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/46cda2cb-ea01-482c-a34d-c27181a010c4-klusterlet-config\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.260703 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.260674    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ch8w\" (UniqueName: \"kubernetes.io/projected/46cda2cb-ea01-482c-a34d-c27181a010c4-kube-api-access-9ch8w\") pod \"klusterlet-addon-workmgr-84f47c546b-wgflk\" (UID: \"46cda2cb-ea01-482c-a34d-c27181a010c4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.350732 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.350652    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:26.350867 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:26.350797    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:26.350867 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:26.350817    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:26.350997 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:26.350882    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:31:42.350865974 +0000 UTC m=+66.807900409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:26.429239 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.429204    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:26.451506 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.451475    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:26.451660 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:26.451633    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:26.451756 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:26.451713    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:31:42.451691636 +0000 UTC m=+66.908726086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:26.587418 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:26.587387    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"]
Jul 28 11:31:26.591445 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:31:26.591422    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46cda2cb_ea01_482c_a34d_c27181a010c4.slice/crio-639a8a62d770bb275668259c7adaee8826ba8dbe662d9b3c0020459fc76a8f93 WatchSource:0}: Error finding container 639a8a62d770bb275668259c7adaee8826ba8dbe662d9b3c0020459fc76a8f93: Status 404 returned error can't find the container with id 639a8a62d770bb275668259c7adaee8826ba8dbe662d9b3c0020459fc76a8f93
Jul 28 11:31:27.258421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:27.258391    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:27.258774 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:27.258491    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:27.258774 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:27.258548    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:31:43.258534644 +0000 UTC m=+67.715569078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:27.396665 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:27.396214    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerStarted","Data":"639a8a62d770bb275668259c7adaee8826ba8dbe662d9b3c0020459fc76a8f93"}
Jul 28 11:31:31.406496 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:31.406452    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerStarted","Data":"294fcf2e5649c7c744031952d445299ff0ff2edac98268a93d3cd2c9d075b48e"}
Jul 28 11:31:31.406982 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:31.406658    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:31.408386 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:31.408365    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:31:31.424801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:31.424757    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podStartSLOduration=1.341838072 podStartE2EDuration="5.424744344s" totalImagesPullingTime="4.082906272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:31.424285891 +0000 UTC m=+55.881320360" watchObservedRunningTime="2026-07-28 11:31:31.424744344 +0000 UTC m=+55.881778779"
Jul 28 11:31:36.337574 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:36.337541    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vnt2w"
Jul 28 11:31:41.774618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.774564    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:31:41.777525 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.777504    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 11:31:41.785545 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:41.785526    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 11:31:41.785624 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:41.785614    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:32:45.785597055 +0000 UTC m=+130.242631490 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : secret "metrics-daemon-secret" not found
Jul 28 11:31:41.875342 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.875294    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:41.875446 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.875407    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:41.877814 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.877769    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzx7h\" (UniqueName: \"kubernetes.io/projected/8f270c87-7947-4f93-af8f-b078821f49f1-kube-api-access-qzx7h\") pod \"network-check-target-2dpkp\" (UID: \"8f270c87-7947-4f93-af8f-b078821f49f1\") " pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:41.877998 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.877981    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 11:31:41.887695 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:41.887672    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d198ec9-287f-46fe-8097-bdb6554c3c6c-original-pull-secret\") pod \"global-pull-secret-syncer-rsxxg\" (UID: \"2d198ec9-287f-46fe-8097-bdb6554c3c6c\") " pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:42.016629 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.016597    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tk4rs\""
Jul 28 11:31:42.024696 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.024642    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:42.028346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.028329    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rsxxg"
Jul 28 11:31:42.159030 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.158998    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rsxxg"]
Jul 28 11:31:42.168435 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.168409    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2dpkp"]
Jul 28 11:31:42.172055 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:31:42.172031    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f270c87_7947_4f93_af8f_b078821f49f1.slice/crio-53fec7ea9d3a1fa9fe212a5c7174701050e91b6b189f64943e97a1ec8487d883 WatchSource:0}: Error finding container 53fec7ea9d3a1fa9fe212a5c7174701050e91b6b189f64943e97a1ec8487d883: Status 404 returned error can't find the container with id 53fec7ea9d3a1fa9fe212a5c7174701050e91b6b189f64943e97a1ec8487d883
Jul 28 11:31:42.379499 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.379409    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:31:42.379668 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:42.379575    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:31:42.379668 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:42.379599    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:31:42.379772 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:42.379672    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:32:14.379643407 +0000 UTC m=+98.836677860 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:31:42.433836 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.433794    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rsxxg" event={"ID":"2d198ec9-287f-46fe-8097-bdb6554c3c6c","Type":"ContainerStarted","Data":"6dba1a656b2efcceb24a7796294c0c953b9c8c6eba448b068678fb20c43c486c"}
Jul 28 11:31:42.435158 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.435132    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2dpkp" event={"ID":"8f270c87-7947-4f93-af8f-b078821f49f1","Type":"ContainerStarted","Data":"dde0bc56cf9357a578c6c6109b136ea4fafebdace25259c96c6db567fded1322"}
Jul 28 11:31:42.435256 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.435178    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2dpkp" event={"ID":"8f270c87-7947-4f93-af8f-b078821f49f1","Type":"ContainerStarted","Data":"53fec7ea9d3a1fa9fe212a5c7174701050e91b6b189f64943e97a1ec8487d883"}
Jul 28 11:31:42.435312 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.435283    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:31:42.450730 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.450679    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2dpkp" podStartSLOduration=66.450664731 podStartE2EDuration="1m6.450664731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:42.450352713 +0000 UTC m=+66.907387170" watchObservedRunningTime="2026-07-28 11:31:42.450664731 +0000 UTC m=+66.907699189"
Jul 28 11:31:42.480336 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:42.480307    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:31:42.480426 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:42.480419    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:31:42.480469 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:42.480463    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:32:14.480452343 +0000 UTC m=+98.937486779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:31:43.287904 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:43.287855    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:31:43.288342 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:43.288022    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:31:43.288342 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:31:43.288091    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:32:15.288072864 +0000 UTC m=+99.745107300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:31:47.450731 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:47.450698    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rsxxg" event={"ID":"2d198ec9-287f-46fe-8097-bdb6554c3c6c","Type":"ContainerStarted","Data":"98d4b1f0e9d41b07c771ec56fccde4652f7aa9afb92decf541cf52091588c5cf"}
Jul 28 11:31:47.466868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:31:47.466817    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rsxxg" podStartSLOduration=67.173845261 podStartE2EDuration="1m11.46680328s" totalImagesPullingTime="4.292958019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:31:47.466317003 +0000 UTC m=+71.923351460" watchObservedRunningTime="2026-07-28 11:31:47.46680328 +0000 UTC m=+71.923837737"
Jul 28 11:32:13.440462 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:32:13.440426    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2dpkp"
Jul 28 11:32:14.425464 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:32:14.425419    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:32:14.425649 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:14.425547    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:32:14.425649 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:14.425560    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:32:14.425649 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:14.425608    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:33:18.42559472 +0000 UTC m=+162.882629155 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:32:14.526329 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:32:14.526289    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:32:14.526703 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:14.526398    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:32:14.526703 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:14.526449    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:33:18.52643421 +0000 UTC m=+162.983468646 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:32:15.332044 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:32:15.331993    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:32:15.332222 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:15.332128    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:32:15.332222 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:15.332180    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:33:19.332166331 +0000 UTC m=+163.789200766 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:32:45.850846 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:32:45.850807    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:32:45.851345 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:45.850957    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 11:32:45.851345 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:32:45.851023    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:34:47.851009258 +0000 UTC m=+252.308043692 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : secret "metrics-daemon-secret" not found
Jul 28 11:33:13.508469 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:13.508422    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e"
Jul 28 11:33:13.562885 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:13.562840    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd" podUID="e625692f-e4a7-46c7-b432-ef63006556ed"
Jul 28 11:33:13.616760 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:13.616726    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9jzkc" podUID="f70ee650-e643-4722-bf6d-cfac90f5859c"
Jul 28 11:33:13.661783 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:13.661756    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:33:13.661923 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:13.661756    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:33:13.661988 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:13.661764    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:33:14.121753 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:14.121718    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:33:18.488378 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:18.488338    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:33:18.488803 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:18.488476    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 11:33:18.488803 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:18.488488    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f54d968f-m6zst: secret "image-registry-tls" not found
Jul 28 11:33:18.488803 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:18.488536    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls podName:891bad0a-f477-4b26-91bb-42fa6104f86e nodeName:}" failed. No retries permitted until 2026-07-28 11:35:20.488522176 +0000 UTC m=+284.945556610 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls") pod "image-registry-76f54d968f-m6zst" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e") : secret "image-registry-tls" not found
Jul 28 11:33:18.589581 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:18.589532    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:33:18.589756 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:18.589713    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 11:33:18.589825 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:18.589809    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls podName:f70ee650-e643-4722-bf6d-cfac90f5859c nodeName:}" failed. No retries permitted until 2026-07-28 11:35:20.589787292 +0000 UTC m=+285.046821731 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls") pod "dns-default-9jzkc" (UID: "f70ee650-e643-4722-bf6d-cfac90f5859c") : secret "dns-default-metrics-tls" not found
Jul 28 11:33:19.394701 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:19.394661    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:33:19.394977 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:19.394779    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 11:33:19.394977 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:33:19.394840    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert podName:e625692f-e4a7-46c7-b432-ef63006556ed nodeName:}" failed. No retries permitted until 2026-07-28 11:35:21.394825831 +0000 UTC m=+285.851860266 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9rfrd" (UID: "e625692f-e4a7-46c7-b432-ef63006556ed") : secret "networking-console-plugin-cert" not found
Jul 28 11:33:26.104103 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:26.104063    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:33:31.407547 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:31.407492    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podUID="46cda2cb-ea01-482c-a34d-c27181a010c4" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused"
Jul 28 11:33:31.708118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:31.708086    2552 generic.go:356] "Generic (PLEG): container finished" podID="46cda2cb-ea01-482c-a34d-c27181a010c4" containerID="294fcf2e5649c7c744031952d445299ff0ff2edac98268a93d3cd2c9d075b48e" exitCode=1
Jul 28 11:33:31.708274 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:31.708135    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerDied","Data":"294fcf2e5649c7c744031952d445299ff0ff2edac98268a93d3cd2c9d075b48e"}
Jul 28 11:33:31.708428 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:31.708415    2552 scope.go:117] "RemoveContainer" containerID="294fcf2e5649c7c744031952d445299ff0ff2edac98268a93d3cd2c9d075b48e"
Jul 28 11:33:32.713688 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:32.713655    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerStarted","Data":"7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"}
Jul 28 11:33:32.714085 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:32.713935    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:33:32.714492 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:33:32.714472    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:34:47.897112 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:47.897062    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:34:47.897597 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:47.897221    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 11:34:47.897597 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:47.897297    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs podName:6364a4d1-df2f-46cd-9f17-335914a957c9 nodeName:}" failed. No retries permitted until 2026-07-28 11:36:49.897281105 +0000 UTC m=+374.354315540 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs") pod "network-metrics-daemon-dqwqh" (UID: "6364a4d1-df2f-46cd-9f17-335914a957c9") : secret "metrics-daemon-secret" not found
Jul 28 11:34:53.050583 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.050546    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"]
Jul 28 11:34:53.053433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.053414    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.057531 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.057513    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 11:34:53.058185 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.058169    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 11:34:53.058804 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.058784    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 11:34:53.059973 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.059946    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hhbjr\""
Jul 28 11:34:53.061273 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.061253    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 11:34:53.074506 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.074485    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"]
Jul 28 11:34:53.132643 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.132609    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.132778 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.132699    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.132778 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.132767    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrt84\" (UniqueName: \"kubernetes.io/projected/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-kube-api-access-nrt84\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.233856 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.233818    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.234042 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.233907    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.234042 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.233978    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrt84\" (UniqueName: \"kubernetes.io/projected/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-kube-api-access-nrt84\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.234347 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.234319    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.236188 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.236165    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.241767 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.241746    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrt84\" (UniqueName: \"kubernetes.io/projected/1b3487ae-02cf-4d41-83c1-f7bfa9e875fa-kube-api-access-nrt84\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr\" (UID: \"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.362070 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.361972    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"
Jul 28 11:34:53.478882 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.478847    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr"]
Jul 28 11:34:53.482545 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:34:53.482516    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3487ae_02cf_4d41_83c1_f7bfa9e875fa.slice/crio-d73ed6e8160302ec975807b57c0ea81a779f33122eae3f1a0636ad332b083fef WatchSource:0}: Error finding container d73ed6e8160302ec975807b57c0ea81a779f33122eae3f1a0636ad332b083fef: Status 404 returned error can't find the container with id d73ed6e8160302ec975807b57c0ea81a779f33122eae3f1a0636ad332b083fef
Jul 28 11:34:53.937354 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:53.937321    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr" event={"ID":"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa","Type":"ContainerStarted","Data":"d73ed6e8160302ec975807b57c0ea81a779f33122eae3f1a0636ad332b083fef"}
Jul 28 11:34:55.944419 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:55.944386    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr" event={"ID":"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa","Type":"ContainerStarted","Data":"0488fd2381c12ba0d56e57b91bb5021349be8ec03ee349a4161d79b330d7820b"}
Jul 28 11:34:55.960671 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:55.960624    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr" podStartSLOduration=1.050444748 podStartE2EDuration="2.960606575s" totalImagesPullingTime="1.910161827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:34:55.960266351 +0000 UTC m=+260.417300809" watchObservedRunningTime="2026-07-28 11:34:55.960606575 +0000 UTC m=+260.417641042"
Jul 28 11:34:56.613410 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.613372    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"]
Jul 28 11:34:56.616487 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.616463    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"
Jul 28 11:34:56.618613 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.618591    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-khllk\""
Jul 28 11:34:56.618709 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.618593    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 11:34:56.618960 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.618943    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 11:34:56.628288 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.628268    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"]
Jul 28 11:34:56.761250 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.761212    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4zfs\" (UniqueName: \"kubernetes.io/projected/c0f29548-3bd8-43c7-a768-0c848ac13d25-kube-api-access-r4zfs\") pod \"migrator-6865c9967-2knl7\" (UID: \"c0f29548-3bd8-43c7-a768-0c848ac13d25\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"
Jul 28 11:34:56.862495 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.862460    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4zfs\" (UniqueName: \"kubernetes.io/projected/c0f29548-3bd8-43c7-a768-0c848ac13d25-kube-api-access-r4zfs\") pod \"migrator-6865c9967-2knl7\" (UID: \"c0f29548-3bd8-43c7-a768-0c848ac13d25\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"
Jul 28 11:34:56.870915 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.870834    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4zfs\" (UniqueName: \"kubernetes.io/projected/c0f29548-3bd8-43c7-a768-0c848ac13d25-kube-api-access-r4zfs\") pod \"migrator-6865c9967-2knl7\" (UID: \"c0f29548-3bd8-43c7-a768-0c848ac13d25\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"
Jul 28 11:34:56.926258 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:56.926229    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"
Jul 28 11:34:57.046419 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.046392    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7"]
Jul 28 11:34:57.049712 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:34:57.049687    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f29548_3bd8_43c7_a768_0c848ac13d25.slice/crio-c17a00f13b56b6c9a118238ae9e6902c0c25453c7b4678476de43a094121798b WatchSource:0}: Error finding container c17a00f13b56b6c9a118238ae9e6902c0c25453c7b4678476de43a094121798b: Status 404 returned error can't find the container with id c17a00f13b56b6c9a118238ae9e6902c0c25453c7b4678476de43a094121798b
Jul 28 11:34:57.870038 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.870000    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ptmkj"]
Jul 28 11:34:57.873352 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.873330    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:57.878307 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.878243    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 11:34:57.878307 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.878279    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 11:34:57.878487 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.878322    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l2vtn\""
Jul 28 11:34:57.878487 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.878416    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 11:34:57.878941 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.878872    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 11:34:57.887724 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.887702    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ptmkj"]
Jul 28 11:34:57.952608 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.952571    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7" event={"ID":"c0f29548-3bd8-43c7-a768-0c848ac13d25","Type":"ContainerStarted","Data":"c17a00f13b56b6c9a118238ae9e6902c0c25453c7b4678476de43a094121798b"}
Jul 28 11:34:57.968758 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.968726    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/771ef471-d88c-44c4-9506-99ff4adc16cf-crio-socket\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:57.968911 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.968785    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:57.968911 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.968825    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:57.968911 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.968843    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49dd5\" (UniqueName: \"kubernetes.io/projected/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-api-access-49dd5\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:57.968911 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:57.968911    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/771ef471-d88c-44c4-9506-99ff4adc16cf-data-volume\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070077 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070040    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49dd5\" (UniqueName: \"kubernetes.io/projected/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-api-access-49dd5\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070127    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/771ef471-d88c-44c4-9506-99ff4adc16cf-data-volume\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070177    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/771ef471-d88c-44c4-9506-99ff4adc16cf-crio-socket\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070227    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070268    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070320    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/771ef471-d88c-44c4-9506-99ff4adc16cf-crio-socket\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.070348    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.070379    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.070432    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:34:58.570412066 +0000 UTC m=+263.027446505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : configmap "kube-rbac-proxy" not found
Jul 28 11:34:58.070448 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.070453    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:34:58.570443603 +0000 UTC m=+263.027478038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:34:58.070744 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.070490    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/771ef471-d88c-44c4-9506-99ff4adc16cf-data-volume\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.079045 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.079024    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49dd5\" (UniqueName: \"kubernetes.io/projected/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-api-access-49dd5\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.573664 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.573564    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.573664 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.573627    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:58.573852 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.573698    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 11:34:58.573852 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.573717    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:34:58.573852 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.573760    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:34:59.573743969 +0000 UTC m=+264.030778404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : configmap "kube-rbac-proxy" not found
Jul 28 11:34:58.573852 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:58.573774    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:34:59.573768791 +0000 UTC m=+264.030803225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:34:58.956609 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.956575    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7" event={"ID":"c0f29548-3bd8-43c7-a768-0c848ac13d25","Type":"ContainerStarted","Data":"248c6f7536821f47291f6162bfa7100eebe0948ba28c8b078482f4d317df5976"}
Jul 28 11:34:58.956770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.956626    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7" event={"ID":"c0f29548-3bd8-43c7-a768-0c848ac13d25","Type":"ContainerStarted","Data":"1926ce35b1047a35b1ef0a6e69b3bb4560a1bf247d1e9afd6893774c60fb4a42"}
Jul 28 11:34:58.982574 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:58.979725    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-2knl7" podStartSLOduration=1.710073028 podStartE2EDuration="2.979706316s" totalImagesPullingTime="1.269633288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:34:58.978722909 +0000 UTC m=+263.435757385" watchObservedRunningTime="2026-07-28 11:34:58.979706316 +0000 UTC m=+263.436740775"
Jul 28 11:34:59.582536 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:59.582493    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:59.582986 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:34:59.582550    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:34:59.582986 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:59.582645    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 11:34:59.582986 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:59.582675    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:34:59.582986 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:59.582720    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:01.582701993 +0000 UTC m=+266.039736428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : configmap "kube-rbac-proxy" not found
Jul 28 11:34:59.582986 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:34:59.582743    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:01.582727548 +0000 UTC m=+266.039761999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:00.283837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.283803    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rsbgd"]
Jul 28 11:35:00.287012 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.286994    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.289056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.289026    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 11:35:00.289149 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.289026    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 11:35:00.289834 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.289820    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 11:35:00.289959 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.289943    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dn6vk\""
Jul 28 11:35:00.290006 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.289953    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 11:35:00.293939 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.293916    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rsbgd"]
Jul 28 11:35:00.388391 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.388361    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cee8e2c-a19e-4222-b259-b48149530016-signing-cabundle\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.388543 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.388441    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtccn\" (UniqueName: \"kubernetes.io/projected/5cee8e2c-a19e-4222-b259-b48149530016-kube-api-access-mtccn\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.388543 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.388466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cee8e2c-a19e-4222-b259-b48149530016-signing-key\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.463768 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.463747    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-74fz4_06c990c3-78d5-4d21-93c9-a474055faa7a/dns-node-resolver/0.log"
Jul 28 11:35:00.489035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.489012    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cee8e2c-a19e-4222-b259-b48149530016-signing-key\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.489111 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.489095    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cee8e2c-a19e-4222-b259-b48149530016-signing-cabundle\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.489148 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.489138    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtccn\" (UniqueName: \"kubernetes.io/projected/5cee8e2c-a19e-4222-b259-b48149530016-kube-api-access-mtccn\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.489714 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.489691    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5cee8e2c-a19e-4222-b259-b48149530016-signing-cabundle\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.491229 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.491208    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5cee8e2c-a19e-4222-b259-b48149530016-signing-key\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.497538 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.497518    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtccn\" (UniqueName: \"kubernetes.io/projected/5cee8e2c-a19e-4222-b259-b48149530016-kube-api-access-mtccn\") pod \"service-ca-658d7b7c78-rsbgd\" (UID: \"5cee8e2c-a19e-4222-b259-b48149530016\") " pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.596326 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.596240    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd"
Jul 28 11:35:00.716869 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.716835    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-rsbgd"]
Jul 28 11:35:00.719817 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:00.719787    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cee8e2c_a19e_4222_b259_b48149530016.slice/crio-01875762cd94104c7910ed3abecb89ce5c249325e81ff1795ff22d028ec0bb81 WatchSource:0}: Error finding container 01875762cd94104c7910ed3abecb89ce5c249325e81ff1795ff22d028ec0bb81: Status 404 returned error can't find the container with id 01875762cd94104c7910ed3abecb89ce5c249325e81ff1795ff22d028ec0bb81
Jul 28 11:35:00.964210 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:00.964175    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd" event={"ID":"5cee8e2c-a19e-4222-b259-b48149530016","Type":"ContainerStarted","Data":"01875762cd94104c7910ed3abecb89ce5c249325e81ff1795ff22d028ec0bb81"}
Jul 28 11:35:01.598816 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:01.598771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:01.599310 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:01.598839    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:01.599310 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:01.598935    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 11:35:01.599310 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:01.598969    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:01.599310 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:01.599025    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:05.598999929 +0000 UTC m=+270.056034387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : configmap "kube-rbac-proxy" not found
Jul 28 11:35:01.599310 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:01.599047    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:05.599036928 +0000 UTC m=+270.056071363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:01.865474 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:01.865393    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r4qzt_f6ed79d9-a93c-4e58-9cf8-b270064850fd/node-ca/0.log"
Jul 28 11:35:02.464498 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:02.464471    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rpzw5_8a63620b-e497-402b-9ea0-864e8f1df157/serve-healthcheck-canary/0.log"
Jul 28 11:35:02.864974 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:02.864941    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-2knl7_c0f29548-3bd8-43c7-a768-0c848ac13d25/migrator/0.log"
Jul 28 11:35:02.972662 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:02.972627    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd" event={"ID":"5cee8e2c-a19e-4222-b259-b48149530016","Type":"ContainerStarted","Data":"2ae9df1158f7d51c06bc1dcaaa28afbb3af8831d8d0a539a3142ac1a2bae9174"}
Jul 28 11:35:02.987337 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:02.987239    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-rsbgd" podStartSLOduration=0.940096386 podStartE2EDuration="2.987220421s" totalImagesPullingTime="2.047124035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:02.987112571 +0000 UTC m=+267.444147032" watchObservedRunningTime="2026-07-28 11:35:02.987220421 +0000 UTC m=+267.444254881"
Jul 28 11:35:03.064343 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:03.064272    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-2knl7_c0f29548-3bd8-43c7-a768-0c848ac13d25/graceful-termination/0.log"
Jul 28 11:35:03.275450 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:03.275412    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr_1b3487ae-02cf-4d41-83c1-f7bfa9e875fa/kube-storage-version-migrator-operator/0.log"
Jul 28 11:35:05.633673 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:05.633627    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:05.634085 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:05.633689    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:05.634085 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:05.633759    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 11:35:05.634085 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:05.633824    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:05.634085 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:05.633826    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:13.633809725 +0000 UTC m=+278.090844160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : configmap "kube-rbac-proxy" not found
Jul 28 11:35:05.634085 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:05.633872    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:13.633861135 +0000 UTC m=+278.090895569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:13.696238 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:13.696200    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:13.696704 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:13.696251    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:13.696704 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:13.696365    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:13.696704 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:13.696451    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls podName:771ef471-d88c-44c4-9506-99ff4adc16cf nodeName:}" failed. No retries permitted until 2026-07-28 11:35:29.696429081 +0000 UTC m=+294.153463517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ptmkj" (UID: "771ef471-d88c-44c4-9506-99ff4adc16cf") : secret "insights-runtime-extractor-tls" not found
Jul 28 11:35:13.696704 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:13.696670    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/771ef471-d88c-44c4-9506-99ff4adc16cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:16.662962 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:16.662920    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd" podUID="e625692f-e4a7-46c7-b432-ef63006556ed"
Jul 28 11:35:16.662962 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:16.662920    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e"
Jul 28 11:35:16.662962 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:16.662920    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9jzkc" podUID="f70ee650-e643-4722-bf6d-cfac90f5859c"
Jul 28 11:35:17.015126 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:17.015093    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:17.015312 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:17.015097    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:17.015312 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:17.015105    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:35:20.552329 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.552289    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:20.554643 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.554622    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"image-registry-76f54d968f-m6zst\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") " pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:20.619433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.619402    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxmgd\""
Jul 28 11:35:20.626444 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.626415    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:20.652951 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.652918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:20.655022 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.655000    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f70ee650-e643-4722-bf6d-cfac90f5859c-metrics-tls\") pod \"dns-default-9jzkc\" (UID: \"f70ee650-e643-4722-bf6d-cfac90f5859c\") " pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:20.763576 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.763540    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:35:20.767444 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:20.767415    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891bad0a_f477_4b26_91bb_42fa6104f86e.slice/crio-990cada89f18bf85d63c72d258dbfc9e0c3e3340981118f4c9c403420d4ffbcf WatchSource:0}: Error finding container 990cada89f18bf85d63c72d258dbfc9e0c3e3340981118f4c9c403420d4ffbcf: Status 404 returned error can't find the container with id 990cada89f18bf85d63c72d258dbfc9e0c3e3340981118f4c9c403420d4ffbcf
Jul 28 11:35:20.918228 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.918188    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l5ffw\""
Jul 28 11:35:20.927060 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:20.927032    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:21.030341 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.029300    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" event={"ID":"891bad0a-f477-4b26-91bb-42fa6104f86e","Type":"ContainerStarted","Data":"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"}
Jul 28 11:35:21.030341 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.029372    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" event={"ID":"891bad0a-f477-4b26-91bb-42fa6104f86e","Type":"ContainerStarted","Data":"990cada89f18bf85d63c72d258dbfc9e0c3e3340981118f4c9c403420d4ffbcf"}
Jul 28 11:35:21.030341 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.030233    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:21.049128 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.048987    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" podStartSLOduration=285.048969387 podStartE2EDuration="4m45.048969387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:21.048301969 +0000 UTC m=+285.505336426" watchObservedRunningTime="2026-07-28 11:35:21.048969387 +0000 UTC m=+285.506003844"
Jul 28 11:35:21.062310 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.062234    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9jzkc"]
Jul 28 11:35:21.065763 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:21.065735    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70ee650_e643_4722_bf6d_cfac90f5859c.slice/crio-811e19765ac999cca44f6851b2a44f7a553957ddeb2b6edaf55c6387b0e48454 WatchSource:0}: Error finding container 811e19765ac999cca44f6851b2a44f7a553957ddeb2b6edaf55c6387b0e48454: Status 404 returned error can't find the container with id 811e19765ac999cca44f6851b2a44f7a553957ddeb2b6edaf55c6387b0e48454
Jul 28 11:35:21.458729 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.458669    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:35:21.461061 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.461033    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e625692f-e4a7-46c7-b432-ef63006556ed-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9rfrd\" (UID: \"e625692f-e4a7-46c7-b432-ef63006556ed\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:35:21.518738 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.518707    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wxnqj\""
Jul 28 11:35:21.527397 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.527374    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"
Jul 28 11:35:21.649803 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:21.649771    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd"]
Jul 28 11:35:21.653322 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:21.653293    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode625692f_e4a7_46c7_b432_ef63006556ed.slice/crio-095123154e6b30bace8d8a47b6d2ae7f2fae7c87a16917f6371f49f7f1a25167 WatchSource:0}: Error finding container 095123154e6b30bace8d8a47b6d2ae7f2fae7c87a16917f6371f49f7f1a25167: Status 404 returned error can't find the container with id 095123154e6b30bace8d8a47b6d2ae7f2fae7c87a16917f6371f49f7f1a25167
Jul 28 11:35:22.033447 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:22.033407    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd" event={"ID":"e625692f-e4a7-46c7-b432-ef63006556ed","Type":"ContainerStarted","Data":"095123154e6b30bace8d8a47b6d2ae7f2fae7c87a16917f6371f49f7f1a25167"}
Jul 28 11:35:22.034449 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:22.034429    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9jzkc" event={"ID":"f70ee650-e643-4722-bf6d-cfac90f5859c","Type":"ContainerStarted","Data":"811e19765ac999cca44f6851b2a44f7a553957ddeb2b6edaf55c6387b0e48454"}
Jul 28 11:35:24.043544 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.043495    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9jzkc" event={"ID":"f70ee650-e643-4722-bf6d-cfac90f5859c","Type":"ContainerStarted","Data":"aa44ceaece62e6fa12d2dcf4dc16f134e1f666bece8971747f9d6af1e9704ca2"}
Jul 28 11:35:24.043999 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.043565    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9jzkc" event={"ID":"f70ee650-e643-4722-bf6d-cfac90f5859c","Type":"ContainerStarted","Data":"69e4a5b973bb4180dd4b02fbc6db02fd7b6f8fd548507e9290ee7114c10a000e"}
Jul 28 11:35:24.043999 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.043596    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:24.044974 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.044949    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd" event={"ID":"e625692f-e4a7-46c7-b432-ef63006556ed","Type":"ContainerStarted","Data":"e7ccdd519481c1c12f0ef0c09fb9ea4c80823fce43ddd4480a207251fe51ce05"}
Jul 28 11:35:24.067217 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.067174    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9jzkc" podStartSLOduration=251.793505648 podStartE2EDuration="4m14.067159245s" totalImagesPullingTime="2.273653597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:24.065642992 +0000 UTC m=+288.522677448" watchObservedRunningTime="2026-07-28 11:35:24.067159245 +0000 UTC m=+288.524193701"
Jul 28 11:35:24.082676 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:24.082637    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9rfrd" podStartSLOduration=254.397058244 podStartE2EDuration="4m16.082624518s" totalImagesPullingTime="1.685566274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:24.081760136 +0000 UTC m=+288.538794604" watchObservedRunningTime="2026-07-28 11:35:24.082624518 +0000 UTC m=+288.539658975"
Jul 28 11:35:27.812585 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.812556    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"]
Jul 28 11:35:27.815515 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.815498    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:27.819126 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.819105    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 11:35:27.819236 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.819112    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-scqfg\""
Jul 28 11:35:27.826908 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.826875    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"]
Jul 28 11:35:27.909488 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:27.909453    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-7kls9\" (UID: \"22915a2d-84a6-409e-89b1-57b48e1bb62e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:28.010307 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:28.010279    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-7kls9\" (UID: \"22915a2d-84a6-409e-89b1-57b48e1bb62e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:28.010463 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:28.010376    2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 11:35:28.010463 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:28.010433    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates podName:22915a2d-84a6-409e-89b1-57b48e1bb62e nodeName:}" failed. No retries permitted until 2026-07-28 11:35:28.510419294 +0000 UTC m=+292.967453730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-7kls9" (UID: "22915a2d-84a6-409e-89b1-57b48e1bb62e") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 11:35:28.514409 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:28.514371    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-7kls9\" (UID: \"22915a2d-84a6-409e-89b1-57b48e1bb62e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:28.516719 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:28.516699    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22915a2d-84a6-409e-89b1-57b48e1bb62e-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-7kls9\" (UID: \"22915a2d-84a6-409e-89b1-57b48e1bb62e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:28.726235 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:28.726193    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:28.859107 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:28.859073    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"]
Jul 28 11:35:28.862095 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:28.862067    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22915a2d_84a6_409e_89b1_57b48e1bb62e.slice/crio-381b0a0a3c43ab9924e38a2f2da23edfb52e8d5e18df82880de6622723db0039 WatchSource:0}: Error finding container 381b0a0a3c43ab9924e38a2f2da23edfb52e8d5e18df82880de6622723db0039: Status 404 returned error can't find the container with id 381b0a0a3c43ab9924e38a2f2da23edfb52e8d5e18df82880de6622723db0039
Jul 28 11:35:29.061276 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:29.061196    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9" event={"ID":"22915a2d-84a6-409e-89b1-57b48e1bb62e","Type":"ContainerStarted","Data":"381b0a0a3c43ab9924e38a2f2da23edfb52e8d5e18df82880de6622723db0039"}
Jul 28 11:35:29.105432 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:29.105402    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dqwqh" podUID="6364a4d1-df2f-46cd-9f17-335914a957c9"
Jul 28 11:35:29.723735 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:29.723689    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:29.726302 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:29.726268    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/771ef471-d88c-44c4-9506-99ff4adc16cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ptmkj\" (UID: \"771ef471-d88c-44c4-9506-99ff4adc16cf\") " pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:29.984511 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:29.984488    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ptmkj"
Jul 28 11:35:30.067909 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:30.067865    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9" event={"ID":"22915a2d-84a6-409e-89b1-57b48e1bb62e","Type":"ContainerStarted","Data":"c2c31cce0a5efe18deff9fb837cfd796ea733e56e16293f9eeebb10000ebc314"}
Jul 28 11:35:30.068072 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:30.068053    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:30.072714 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:30.072690    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9"
Jul 28 11:35:30.087948 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:30.087881    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-7kls9" podStartSLOduration=2.019901067 podStartE2EDuration="3.087865603s" totalImagesPullingTime="1.067964536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:30.08779257 +0000 UTC m=+294.544827026" watchObservedRunningTime="2026-07-28 11:35:30.087865603 +0000 UTC m=+294.544900061"
Jul 28 11:35:30.124455 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:30.124422    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ptmkj"]
Jul 28 11:35:30.129858 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:30.129829    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771ef471_d88c_44c4_9506_99ff4adc16cf.slice/crio-2b5f1fb519307f7650e227f60d357f2b5e0286aa8ef99d5a11e951eda63131f7 WatchSource:0}: Error finding container 2b5f1fb519307f7650e227f60d357f2b5e0286aa8ef99d5a11e951eda63131f7: Status 404 returned error can't find the container with id 2b5f1fb519307f7650e227f60d357f2b5e0286aa8ef99d5a11e951eda63131f7
Jul 28 11:35:31.073760 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:31.073667    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptmkj" event={"ID":"771ef471-d88c-44c4-9506-99ff4adc16cf","Type":"ContainerStarted","Data":"2b5f1fb519307f7650e227f60d357f2b5e0286aa8ef99d5a11e951eda63131f7"}
Jul 28 11:35:32.079821 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:32.079742    2552 generic.go:356] "Generic (PLEG): container finished" podID="46cda2cb-ea01-482c-a34d-c27181a010c4" containerID="7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51" exitCode=1
Jul 28 11:35:32.080243 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:32.079822    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerDied","Data":"7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"}
Jul 28 11:35:32.080243 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:32.079911    2552 scope.go:117] "RemoveContainer" containerID="294fcf2e5649c7c744031952d445299ff0ff2edac98268a93d3cd2c9d075b48e"
Jul 28 11:35:32.080382 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:32.080368    2552 scope.go:117] "RemoveContainer" containerID="7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"
Jul 28 11:35:32.080588 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:32.080550    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-84f47c546b-wgflk_open-cluster-management-agent-addon(46cda2cb-ea01-482c-a34d-c27181a010c4)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podUID="46cda2cb-ea01-482c-a34d-c27181a010c4"
Jul 28 11:35:32.714573 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:32.714526    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:35:33.085534 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:33.085462    2552 scope.go:117] "RemoveContainer" containerID="7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"
Jul 28 11:35:33.085992 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:33.085617    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-84f47c546b-wgflk_open-cluster-management-agent-addon(46cda2cb-ea01-482c-a34d-c27181a010c4)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podUID="46cda2cb-ea01-482c-a34d-c27181a010c4"
Jul 28 11:35:34.051695 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:34.051664    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9jzkc"
Jul 28 11:35:35.980747 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:35.980722    2552 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 11:35:36.097124 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:36.096966    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptmkj" event={"ID":"771ef471-d88c-44c4-9506-99ff4adc16cf","Type":"ContainerStarted","Data":"05e619fc9a3303c62ab8f8483bd17fe5bd3af198c2c8ae869b07b23827c6740a"}
Jul 28 11:35:36.429646 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:36.429570    2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:35:36.430023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:36.430008    2552 scope.go:117] "RemoveContainer" containerID="7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"
Jul 28 11:35:36.430191 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:36.430169    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-84f47c546b-wgflk_open-cluster-management-agent-addon(46cda2cb-ea01-482c-a34d-c27181a010c4)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podUID="46cda2cb-ea01-482c-a34d-c27181a010c4"
Jul 28 11:35:38.264402 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.264360    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qh496"]
Jul 28 11:35:38.278172 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.278137    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.280660 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.280489    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281125    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281138    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281307    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281362    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281550    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gnbvz\""
Jul 28 11:35:38.281742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.281551    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 11:35:38.400020 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.399980    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400196 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400045    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-wtmp\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400196 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400114    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-root\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400196 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400187    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6lvj\" (UniqueName: \"kubernetes.io/projected/a6feb4df-94c1-429a-88be-0c18fb25bc08-kube-api-access-b6lvj\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400355 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400215    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-textfile\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400355 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400245    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-sys\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400355 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400290    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-metrics-client-ca\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400355 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400312    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-tls\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.400355 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.400344    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501599 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501573    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6lvj\" (UniqueName: \"kubernetes.io/projected/a6feb4df-94c1-429a-88be-0c18fb25bc08-kube-api-access-b6lvj\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501733 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-textfile\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501733 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501645    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-sys\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501733 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501701    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-metrics-client-ca\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501733 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501726    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-tls\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501764    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501804    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501854    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-wtmp\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.501965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501905    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-root\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.502151 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.501990    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-root\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.502151 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.502026    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-textfile\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.502399 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.502242    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-wtmp\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.502399 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.502300    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6feb4df-94c1-429a-88be-0c18fb25bc08-sys\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.502754 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.502727    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-accelerators-collector-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.503209 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.503185    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6feb4df-94c1-429a-88be-0c18fb25bc08-metrics-client-ca\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.504542 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.504515    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-tls\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.504621 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.504564    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6feb4df-94c1-429a-88be-0c18fb25bc08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.509523 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.509499    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6lvj\" (UniqueName: \"kubernetes.io/projected/a6feb4df-94c1-429a-88be-0c18fb25bc08-kube-api-access-b6lvj\") pod \"node-exporter-qh496\" (UID: \"a6feb4df-94c1-429a-88be-0c18fb25bc08\") " pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.571638 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.571618    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:35:38.589262 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:38.589235    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qh496"
Jul 28 11:35:38.600077 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:38.600047    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6feb4df_94c1_429a_88be_0c18fb25bc08.slice/crio-49432d9c5ceea3287493c5f5e8e07796aa796d679d4cc3213525f18eed6eeb8b WatchSource:0}: Error finding container 49432d9c5ceea3287493c5f5e8e07796aa796d679d4cc3213525f18eed6eeb8b: Status 404 returned error can't find the container with id 49432d9c5ceea3287493c5f5e8e07796aa796d679d4cc3213525f18eed6eeb8b
Jul 28 11:35:39.111115 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.111070    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh496" event={"ID":"a6feb4df-94c1-429a-88be-0c18fb25bc08","Type":"ContainerStarted","Data":"49432d9c5ceea3287493c5f5e8e07796aa796d679d4cc3213525f18eed6eeb8b"}
Jul 28 11:35:39.113846 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.113809    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptmkj" event={"ID":"771ef471-d88c-44c4-9506-99ff4adc16cf","Type":"ContainerStarted","Data":"1a2fb9a82cc4c2e33b513ae72b0084b7df1ceca689688a5012919f4b3aab8261"}
Jul 28 11:35:39.362873 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.362788    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:35:39.367266 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.367232    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.370064 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370035    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 11:35:39.370306 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370241    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ggs5\""
Jul 28 11:35:39.370306 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370291    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 11:35:39.370452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370411    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 11:35:39.370591 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370246    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 11:35:39.370717 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.370641    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 11:35:39.371199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.371179    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 11:35:39.373061 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.371430    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 11:35:39.373061 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.371476    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 11:35:39.373061 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.371684    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 11:35:39.385791 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.385764    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:35:39.409971 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.409941    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mbtx\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410081 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410001    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410081 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410030    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410081 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410053    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410081 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410076    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410105    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410129    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410154    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410203    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410234    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410267    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410298    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.410661 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.410324    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.511783 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mbtx\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511817    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511849    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511913    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511945    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511967    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512013 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.511995    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512044    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512078    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512114    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512145    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512173    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512483 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:39.512362    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle podName:729c0887-ad3a-4938-ba45-dd4ad7f55917 nodeName:}" failed. No retries permitted until 2026-07-28 11:35:40.01234446 +0000 UTC m=+304.469378895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917") : configmap references non-existent config key: ca-bundle.crt
Jul 28 11:35:39.512812 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512567    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512812 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:39.512639    2552 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 11:35:39.512812 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.512645    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.512812 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:39.512692    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls podName:729c0887-ad3a-4938-ba45-dd4ad7f55917 nodeName:}" failed. No retries permitted until 2026-07-28 11:35:40.012675041 +0000 UTC m=+304.469709476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917") : secret "alertmanager-main-tls" not found
Jul 28 11:35:39.516925 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.515864    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.517200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.517136    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.518955 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.518315    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.519188 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.519152    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.519503 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.519469    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.520409 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.520367    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.520499 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.520473    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.522208 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.522184    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:39.523031 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:39.523008    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mbtx\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.015971 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.015932    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.016153 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.015995    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.016867 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.016840    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.018703 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.018674    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.119446 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.119408    2552 generic.go:356] "Generic (PLEG): container finished" podID="a6feb4df-94c1-429a-88be-0c18fb25bc08" containerID="31bab5c83000a7d91dc271ed933ec0fb8f1dd4816c3f9cf6ab34862fe076cd3b" exitCode=0
Jul 28 11:35:40.119613 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.119461    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh496" event={"ID":"a6feb4df-94c1-429a-88be-0c18fb25bc08","Type":"ContainerDied","Data":"31bab5c83000a7d91dc271ed933ec0fb8f1dd4816c3f9cf6ab34862fe076cd3b"}
Jul 28 11:35:40.280961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.280875    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:35:40.431699 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.431670    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:35:40.631805 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.631718    2552 patch_prober.go:28] interesting pod/image-registry-76f54d968f-m6zst container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 11:35:40.631805 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:40.631772    2552 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 11:35:41.126022 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.125986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh496" event={"ID":"a6feb4df-94c1-429a-88be-0c18fb25bc08","Type":"ContainerStarted","Data":"47b0fb9e418ba55b54b14f52759ca748e747ef699e60bf616f7ab4e08614f794"}
Jul 28 11:35:41.126203 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.126048    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qh496" event={"ID":"a6feb4df-94c1-429a-88be-0c18fb25bc08","Type":"ContainerStarted","Data":"26c38ece952b1b5429c38fa2c385ae1653f15b8187654c16d104996d2f88f435"}
Jul 28 11:35:41.144490 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.144426    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qh496" podStartSLOduration=2.064495041 podStartE2EDuration="3.144411237s" totalImagesPullingTime="1.003854995s" totalInitContainerRuntime="76.061201ms" isStatefulPod=true podCreationTimestamp="2026-07-28 11:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:41.143806929 +0000 UTC m=+305.600841388" watchObservedRunningTime="2026-07-28 11:35:41.144411237 +0000 UTC m=+305.601445692"
Jul 28 11:35:41.255688 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.255628    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"]
Jul 28 11:35:41.272076 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.272040    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274292    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"]
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274429    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274578    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274653    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w984w\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274860    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.274950    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.275121    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9bom7r74dqchv\""
Jul 28 11:35:41.275328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.275167    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 11:35:41.327538 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327492    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ad7485-4bc8-4086-82ea-16b2e98157b7-metrics-client-ca\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327705 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327551    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327705 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327652    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327753    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-grpc-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327783    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.327854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327832    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.328093 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.327869    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-986ns\" (UniqueName: \"kubernetes.io/projected/70ad7485-4bc8-4086-82ea-16b2e98157b7-kube-api-access-986ns\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428574 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428484    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428574 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428546    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428576    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-grpc-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428609    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428657    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428683    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-986ns\" (UniqueName: \"kubernetes.io/projected/70ad7485-4bc8-4086-82ea-16b2e98157b7-kube-api-access-986ns\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428724    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ad7485-4bc8-4086-82ea-16b2e98157b7-metrics-client-ca\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.428794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.428767    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.429939 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.429911    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70ad7485-4bc8-4086-82ea-16b2e98157b7-metrics-client-ca\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.431742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.431709    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.433405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.431860    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.433405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.432434    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-grpc-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.433405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.433026    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.433727 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.433677    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.434331 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.434299    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/70ad7485-4bc8-4086-82ea-16b2e98157b7-secret-thanos-querier-tls\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.438519 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.438495    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-986ns\" (UniqueName: \"kubernetes.io/projected/70ad7485-4bc8-4086-82ea-16b2e98157b7-kube-api-access-986ns\") pod \"thanos-querier-6f4bb6cd74-ks5l5\" (UID: \"70ad7485-4bc8-4086-82ea-16b2e98157b7\") " pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:41.585006 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:41.584966    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:42.103392 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.103354    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:35:42.137230 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:42.137199    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729c0887_ad3a_4938_ba45_dd4ad7f55917.slice/crio-a242fa827e5e47093ee0558d9143a6f50d9638ca14f048a9239f5df98afffe70 WatchSource:0}: Error finding container a242fa827e5e47093ee0558d9143a6f50d9638ca14f048a9239f5df98afffe70: Status 404 returned error can't find the container with id a242fa827e5e47093ee0558d9143a6f50d9638ca14f048a9239f5df98afffe70
Jul 28 11:35:42.269499 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.269470    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"]
Jul 28 11:35:42.273002 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:42.272979    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ad7485_4bc8_4086_82ea_16b2e98157b7.slice/crio-1d5383c591b2f4275dbcf0e272cfdf561837349a59dfa22a9fd88dcb3f5ebb16 WatchSource:0}: Error finding container 1d5383c591b2f4275dbcf0e272cfdf561837349a59dfa22a9fd88dcb3f5ebb16: Status 404 returned error can't find the container with id 1d5383c591b2f4275dbcf0e272cfdf561837349a59dfa22a9fd88dcb3f5ebb16
Jul 28 11:35:42.579636 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.579599    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bd9489dd7-mpjxc"]
Jul 28 11:35:42.582905 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.582881    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.586926 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.586890    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lfbw8\""
Jul 28 11:35:42.587054 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.587027    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2p5k90pkf2i7j\""
Jul 28 11:35:42.587054 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.587037    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 11:35:42.587184 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.587114    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 11:35:42.587184 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.587129    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 11:35:42.587298 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.587283    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 11:35:42.594426 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.594402    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bd9489dd7-mpjxc"]
Jul 28 11:35:42.642168 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642137    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-client-certs\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642314 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642177    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-tls\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642314 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642205    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3b77eabb-3a87-4281-a736-5ab554dd3b92-audit-log\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642314 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642275    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-client-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642453 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642316    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-metrics-server-audit-profiles\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642453 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642375    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.642530 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.642468    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n2bx\" (UniqueName: \"kubernetes.io/projected/3b77eabb-3a87-4281-a736-5ab554dd3b92-kube-api-access-2n2bx\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743070 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743037    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n2bx\" (UniqueName: \"kubernetes.io/projected/3b77eabb-3a87-4281-a736-5ab554dd3b92-kube-api-access-2n2bx\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743228 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743083    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-client-certs\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743228 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743113    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-tls\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743228 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743145    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3b77eabb-3a87-4281-a736-5ab554dd3b92-audit-log\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743228 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743197    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-client-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743479 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743240    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-metrics-server-audit-profiles\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743479 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743273    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.743989 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.743876    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3b77eabb-3a87-4281-a736-5ab554dd3b92-audit-log\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.744123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.744054    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.744645 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.744616    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3b77eabb-3a87-4281-a736-5ab554dd3b92-metrics-server-audit-profiles\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.746290 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.746264    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-client-ca-bundle\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.746390 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.746294    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-tls\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.746390 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.746337    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3b77eabb-3a87-4281-a736-5ab554dd3b92-secret-metrics-server-client-certs\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.752271 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.752247    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n2bx\" (UniqueName: \"kubernetes.io/projected/3b77eabb-3a87-4281-a736-5ab554dd3b92-kube-api-access-2n2bx\") pod \"metrics-server-bd9489dd7-mpjxc\" (UID: \"3b77eabb-3a87-4281-a736-5ab554dd3b92\") " pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:42.892476 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:42.892386    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:35:43.030216 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.030176    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"]
Jul 28 11:35:43.033715 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.033689    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.036159 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.035795    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 11:35:43.036159 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.036001    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xg2br\""
Jul 28 11:35:43.043351 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.043323    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"]
Jul 28 11:35:43.045366 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.045344    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:35:43.135212 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.135176    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ptmkj" event={"ID":"771ef471-d88c-44c4-9506-99ff4adc16cf","Type":"ContainerStarted","Data":"6daea664e911df298d488ba920ea28cc38068283e0b30fc085d8816cf7547cbd"}
Jul 28 11:35:43.138751 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.138642    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"1d5383c591b2f4275dbcf0e272cfdf561837349a59dfa22a9fd88dcb3f5ebb16"}
Jul 28 11:35:43.140598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.140570    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"a242fa827e5e47093ee0558d9143a6f50d9638ca14f048a9239f5df98afffe70"}
Jul 28 11:35:43.146327 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.146261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sql6g\" (UniqueName: \"kubernetes.io/projected/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-kube-api-access-sql6g\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.146327 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.146304    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.154174 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.154121    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ptmkj" podStartSLOduration=34.238837072 podStartE2EDuration="46.154102449s" totalImagesPullingTime="11.915265377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:34:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:43.152733157 +0000 UTC m=+307.609767656" watchObservedRunningTime="2026-07-28 11:35:43.154102449 +0000 UTC m=+307.611136906"
Jul 28 11:35:43.247433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.246990    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sql6g\" (UniqueName: \"kubernetes.io/projected/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-kube-api-access-sql6g\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.247433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.247029    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.247433 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:43.247330    2552 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 11:35:43.247433 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:35:43.247389    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert podName:0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc nodeName:}" failed. No retries permitted until 2026-07-28 11:35:43.747371104 +0000 UTC m=+308.204405554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-jsk9x" (UID: "0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc") : secret "monitoring-plugin-cert" not found
Jul 28 11:35:43.265875 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.265599    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sql6g\" (UniqueName: \"kubernetes.io/projected/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-kube-api-access-sql6g\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.349053 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.349016    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bd9489dd7-mpjxc"]
Jul 28 11:35:43.352722 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:43.352695    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b77eabb_3a87_4281_a736_5ab554dd3b92.slice/crio-3a567e0cc64226d56630d38006f914a7227e683ca4cb42abfd1db845e275f698 WatchSource:0}: Error finding container 3a567e0cc64226d56630d38006f914a7227e683ca4cb42abfd1db845e275f698: Status 404 returned error can't find the container with id 3a567e0cc64226d56630d38006f914a7227e683ca4cb42abfd1db845e275f698
Jul 28 11:35:43.752363 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.752320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.755045 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.755020    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jsk9x\" (UID: \"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:43.946422 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:43.946383    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:44.147552 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.147510    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc" event={"ID":"3b77eabb-3a87-4281-a736-5ab554dd3b92","Type":"ContainerStarted","Data":"3a567e0cc64226d56630d38006f914a7227e683ca4cb42abfd1db845e275f698"}
Jul 28 11:35:44.149612 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.149532    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87" exitCode=0
Jul 28 11:35:44.149612 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.149575    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87"}
Jul 28 11:35:44.216267 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.216238    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"]
Jul 28 11:35:44.220620 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:35:44.220592    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0852eab5_b13b_4ec4_9b26_f7aaae7ca8dc.slice/crio-c1f64de08b556e231201cb96f5ac2725434623f50b5eee3b3b4f8e71e1caeee2 WatchSource:0}: Error finding container c1f64de08b556e231201cb96f5ac2725434623f50b5eee3b3b4f8e71e1caeee2: Status 404 returned error can't find the container with id c1f64de08b556e231201cb96f5ac2725434623f50b5eee3b3b4f8e71e1caeee2
Jul 28 11:35:44.444514 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.444476    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:35:44.907833 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.907697    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:35:44.911312 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.911280    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:44.913570 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.913476    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dzxgt\""
Jul 28 11:35:44.913570 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.913484    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 11:35:44.913733 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.913707    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 11:35:44.913810 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.913754    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 11:35:44.914060 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.914038    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 11:35:44.914173 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.914093    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 11:35:44.914173 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.914041    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 11:35:44.914173 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.914127    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 11:35:44.919397 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:44.919377    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:35:45.064653 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064612    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.064817 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064712    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.064817 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064746    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.064817 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064806    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2znz\" (UniqueName: \"kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.065023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064838    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.065023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.064926    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.154402 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.154370    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x" event={"ID":"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc","Type":"ContainerStarted","Data":"c1f64de08b556e231201cb96f5ac2725434623f50b5eee3b3b4f8e71e1caeee2"}
Jul 28 11:35:45.158050 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.157986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"fbb8be09431016eb8bf625d500fc0d219c576767f8fca41d5d2373ae43e2fbc4"}
Jul 28 11:35:45.158149 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.158059    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"d71fa3d6324419afe115821d6df948ad84f760004a34fba12eca16a88b8dfcb2"}
Jul 28 11:35:45.158149 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.158104    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"53f58a4896d2fb707efff3cc84a96022a3529e021ac9ba0c267e039e588a7418"}
Jul 28 11:35:45.165395 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165352    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.165504 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165397    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.165504 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165428    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2znz\" (UniqueName: \"kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.165604 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.165665 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165628    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.165721 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.165688    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.166396 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.166328    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.166514 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.166411    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.166514 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.166471    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.168678 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.168653    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.168678 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.168636    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.175516 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.175493    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2znz\" (UniqueName: \"kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz\") pod \"console-64bd945456-wkrbp\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") " pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.225254 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.225229    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:45.386943 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:45.386680    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:35:46.165504 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:46.165457    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc" event={"ID":"3b77eabb-3a87-4281-a736-5ab554dd3b92","Type":"ContainerStarted","Data":"864de88931a110c943c23eb9ee7fa56e6eb1f7a128e0be9fa0d2cff80342dd0f"}
Jul 28 11:35:46.166979 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:46.166947    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64bd945456-wkrbp" event={"ID":"c1e66f8c-85eb-47b4-894d-9f1e172bfb32","Type":"ContainerStarted","Data":"5fbd003eb429f07749891b1d392ad9800bc470238ce682786a9af9be74763c01"}
Jul 28 11:35:46.185082 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:46.185038    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc" podStartSLOduration=2.392070617 podStartE2EDuration="4.185022158s" totalImagesPullingTime="1.792951541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:46.184435221 +0000 UTC m=+310.641469679" watchObservedRunningTime="2026-07-28 11:35:46.185022158 +0000 UTC m=+310.642056593"
Jul 28 11:35:47.176516 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176463    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"}
Jul 28 11:35:47.177010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176595    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"}
Jul 28 11:35:47.177010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176643    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"}
Jul 28 11:35:47.177010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176687    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"}
Jul 28 11:35:47.177010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176730    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"}
Jul 28 11:35:47.177010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.176771    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerStarted","Data":"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"}
Jul 28 11:35:47.178938 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.178880    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x" event={"ID":"0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc","Type":"ContainerStarted","Data":"49bd611c375ae66366363f683f798d9105e963e7017933171f1ee50ba7bf4cb1"}
Jul 28 11:35:47.179142 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.179100    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:47.183982 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.183956    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"6388444bdc8bf852169ef4aa4c9fa52ebd7cb335df54d511fc4774ac0ca91974"}
Jul 28 11:35:47.184079 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.184030    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"39e55f0876703025c2fedbafbab0c8e91426c3935c4bd68fe0d8961c31cd8061"}
Jul 28 11:35:47.184079 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.184075    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" event={"ID":"70ad7485-4bc8-4086-82ea-16b2e98157b7","Type":"ContainerStarted","Data":"5df81571609de34796f95f38cd1a451688091691b5058e5c42cad11180504bcd"}
Jul 28 11:35:47.185639 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.185596    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x"
Jul 28 11:35:47.205128 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.205082    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.785838301 podStartE2EDuration="8.205066366s" totalImagesPullingTime="3.398039258s" totalInitContainerRuntime="21.188807ms" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:47.202521004 +0000 UTC m=+311.659555462" watchObservedRunningTime="2026-07-28 11:35:47.205066366 +0000 UTC m=+311.662100822"
Jul 28 11:35:47.228997 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.228939    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5" podStartSLOduration=2.28991498 podStartE2EDuration="6.228921287s" totalImagesPullingTime="3.939006307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:47.227792805 +0000 UTC m=+311.684827286" watchObservedRunningTime="2026-07-28 11:35:47.228921287 +0000 UTC m=+311.685955744"
Jul 28 11:35:47.242005 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:47.241950    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jsk9x" podStartSLOduration=2.022630573 podStartE2EDuration="4.241930754s" totalImagesPullingTime="2.219300181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:47.241187909 +0000 UTC m=+311.698222368" watchObservedRunningTime="2026-07-28 11:35:47.241930754 +0000 UTC m=+311.698965213"
Jul 28 11:35:48.189453 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:48.189415    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:49.193593 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:49.193546    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64bd945456-wkrbp" event={"ID":"c1e66f8c-85eb-47b4-894d-9f1e172bfb32","Type":"ContainerStarted","Data":"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"}
Jul 28 11:35:49.201168 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:49.201143    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f4bb6cd74-ks5l5"
Jul 28 11:35:49.219932 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:49.218087    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64bd945456-wkrbp" podStartSLOduration=1.940645817 podStartE2EDuration="5.21806968s" totalImagesPullingTime="3.277423863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:35:49.216910757 +0000 UTC m=+313.673945206" watchObservedRunningTime="2026-07-28 11:35:49.21806968 +0000 UTC m=+313.675104138"
Jul 28 11:35:50.103453 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:50.103417    2552 scope.go:117] "RemoveContainer" containerID="7349bd253dcf353346347200a841e8e734870c2162620e3dcf52caf22e3dae51"
Jul 28 11:35:50.199186 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:50.199156    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" event={"ID":"46cda2cb-ea01-482c-a34d-c27181a010c4","Type":"ContainerStarted","Data":"d05c32a1694e2b4844828d3a70aef318382bdcb0cb7e7332501ebb30bda83b8c"}
Jul 28 11:35:50.199684 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:50.199662    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:35:50.201098 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:50.201069    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk" podUID="46cda2cb-ea01-482c-a34d-c27181a010c4" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused"
Jul 28 11:35:51.204275 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:51.204249    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84f47c546b-wgflk"
Jul 28 11:35:55.226143 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:55.226104    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:55.226675 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:55.226167    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:55.231046 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:55.231020    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:35:56.226481 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:35:56.226444    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:36:02.893219 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:02.893179    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:36:02.893592 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:02.893240    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:36:06.401625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:06.401591    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:36:09.469851 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.469811    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerName="registry" containerID="cri-o://05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c" gracePeriod=30
Jul 28 11:36:09.767763 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.767734    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:36:09.777024 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777000    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777030    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777049    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777086    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777134    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4pmh\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777167    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777214    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.777400 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777259    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") pod \"891bad0a-f477-4b26-91bb-42fa6104f86e\" (UID: \"891bad0a-f477-4b26-91bb-42fa6104f86e\") "
Jul 28 11:36:09.778071 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.777849    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:09.778176 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.778102    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:09.779522 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.779501    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:09.779631 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.779610    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:09.779861 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.779831    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:09.779967 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.779874    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh" (OuterVolumeSpecName: "kube-api-access-n4pmh") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "kube-api-access-n4pmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:09.780011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.779977    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:09.786279 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.786254    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "891bad0a-f477-4b26-91bb-42fa6104f86e" (UID: "891bad0a-f477-4b26-91bb-42fa6104f86e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:09.878500 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878462    2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-installation-pull-secrets\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878500 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878493    2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-certificates\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878500 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878505    2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/891bad0a-f477-4b26-91bb-42fa6104f86e-ca-trust-extracted\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878520    2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/891bad0a-f477-4b26-91bb-42fa6104f86e-image-registry-private-configuration\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878530    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4pmh\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-kube-api-access-n4pmh\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878538    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/891bad0a-f477-4b26-91bb-42fa6104f86e-trusted-ca\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878546    2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-bound-sa-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:09.878723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:09.878554    2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/891bad0a-f477-4b26-91bb-42fa6104f86e-registry-tls\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:10.275154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.275114    2552 generic.go:356] "Generic (PLEG): container finished" podID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerID="05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c" exitCode=0
Jul 28 11:36:10.275323 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.275185    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f54d968f-m6zst"
Jul 28 11:36:10.275323 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.275196    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" event={"ID":"891bad0a-f477-4b26-91bb-42fa6104f86e","Type":"ContainerDied","Data":"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"}
Jul 28 11:36:10.275323 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.275248    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f54d968f-m6zst" event={"ID":"891bad0a-f477-4b26-91bb-42fa6104f86e","Type":"ContainerDied","Data":"990cada89f18bf85d63c72d258dbfc9e0c3e3340981118f4c9c403420d4ffbcf"}
Jul 28 11:36:10.275323 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.275269    2552 scope.go:117] "RemoveContainer" containerID="05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"
Jul 28 11:36:10.284979 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.284961    2552 scope.go:117] "RemoveContainer" containerID="05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"
Jul 28 11:36:10.285306 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:36:10.285283    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c\": container with ID starting with 05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c not found: ID does not exist" containerID="05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"
Jul 28 11:36:10.285381 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.285318    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c"} err="failed to get container status \"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c\": rpc error: code = NotFound desc = could not find container \"05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c\": container with ID starting with 05954a827e1461e89e48f607fe87d31b0a4ed06a92ce1065ac9f60065ee3a78c not found: ID does not exist"
Jul 28 11:36:10.297749 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.297722    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:36:10.312592 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:10.312566    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76f54d968f-m6zst"]
Jul 28 11:36:12.107629 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:12.107597    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" path="/var/lib/kubelet/pods/891bad0a-f477-4b26-91bb-42fa6104f86e/volumes"
Jul 28 11:36:21.319343 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:21.319253    2552 generic.go:356] "Generic (PLEG): container finished" podID="1b3487ae-02cf-4d41-83c1-f7bfa9e875fa" containerID="0488fd2381c12ba0d56e57b91bb5021349be8ec03ee349a4161d79b330d7820b" exitCode=0
Jul 28 11:36:21.319748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:21.319334    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr" event={"ID":"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa","Type":"ContainerDied","Data":"0488fd2381c12ba0d56e57b91bb5021349be8ec03ee349a4161d79b330d7820b"}
Jul 28 11:36:21.319748 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:21.319647    2552 scope.go:117] "RemoveContainer" containerID="0488fd2381c12ba0d56e57b91bb5021349be8ec03ee349a4161d79b330d7820b"
Jul 28 11:36:22.325534 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:22.325498    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr" event={"ID":"1b3487ae-02cf-4d41-83c1-f7bfa9e875fa","Type":"ContainerStarted","Data":"c361a4d96b530b2a2647834760f19cacccbabde94a56a9bd3c5683c9d69da131"}
Jul 28 11:36:22.898073 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:22.898044    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:36:22.901782 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:22.901758    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bd9489dd7-mpjxc"
Jul 28 11:36:31.420243 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.420148    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64bd945456-wkrbp" podUID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" containerName="console" containerID="cri-o://000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b" gracePeriod=15
Jul 28 11:36:31.662731 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.662703    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:36:31.759624 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759593    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.759801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759638    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.759801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759661    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.759801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759678    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.759801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759701    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2znz\" (UniqueName: \"kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.759801 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.759726    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config\") pod \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\" (UID: \"c1e66f8c-85eb-47b4-894d-9f1e172bfb32\") "
Jul 28 11:36:31.760194 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.760163    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config" (OuterVolumeSpecName: "console-config") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:31.760194 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.760180    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca" (OuterVolumeSpecName: "service-ca") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:31.760356 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.760221    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:31.762003 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.761972    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:31.762111 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.762082    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:31.762161 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.762146    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz" (OuterVolumeSpecName: "kube-api-access-n2znz") pod "c1e66f8c-85eb-47b4-894d-9f1e172bfb32" (UID: "c1e66f8c-85eb-47b4-894d-9f1e172bfb32"). InnerVolumeSpecName "kube-api-access-n2znz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:31.861023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.860984    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:31.861023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.861020    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-oauth-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:31.861023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.861030    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-service-ca\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:31.861023 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.861039    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:31.861270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.861047    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2znz\" (UniqueName: \"kubernetes.io/projected/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-kube-api-access-n2znz\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:31.861270 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:31.861055    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c1e66f8c-85eb-47b4-894d-9f1e172bfb32-console-oauth-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:32.366868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.366778    2552 generic.go:356] "Generic (PLEG): container finished" podID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" containerID="000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b" exitCode=0
Jul 28 11:36:32.366868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.366823    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64bd945456-wkrbp"
Jul 28 11:36:32.366868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.366842    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64bd945456-wkrbp" event={"ID":"c1e66f8c-85eb-47b4-894d-9f1e172bfb32","Type":"ContainerDied","Data":"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"}
Jul 28 11:36:32.367100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.366881    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64bd945456-wkrbp" event={"ID":"c1e66f8c-85eb-47b4-894d-9f1e172bfb32","Type":"ContainerDied","Data":"5fbd003eb429f07749891b1d392ad9800bc470238ce682786a9af9be74763c01"}
Jul 28 11:36:32.367100 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.366921    2552 scope.go:117] "RemoveContainer" containerID="000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"
Jul 28 11:36:32.375085 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.375066    2552 scope.go:117] "RemoveContainer" containerID="000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"
Jul 28 11:36:32.375339 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:36:32.375317    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b\": container with ID starting with 000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b not found: ID does not exist" containerID="000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"
Jul 28 11:36:32.375404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.375351    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b"} err="failed to get container status \"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b\": rpc error: code = NotFound desc = could not find container \"000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b\": container with ID starting with 000bcf2588c327130cb9888f452776a641a1f879cc198ae5a58793eba384000b not found: ID does not exist"
Jul 28 11:36:32.407871 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.407838    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:36:32.415779 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:32.415752    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64bd945456-wkrbp"]
Jul 28 11:36:34.108252 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:34.108211    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" path="/var/lib/kubelet/pods/c1e66f8c-85eb-47b4-894d-9f1e172bfb32/volumes"
Jul 28 11:36:49.910658 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:49.910619    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:36:49.912952 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:49.912929    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6364a4d1-df2f-46cd-9f17-335914a957c9-metrics-certs\") pod \"network-metrics-daemon-dqwqh\" (UID: \"6364a4d1-df2f-46cd-9f17-335914a957c9\") " pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:36:50.206757 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:50.206726    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2mvc2\""
Jul 28 11:36:50.215098 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:50.215079    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dqwqh"
Jul 28 11:36:50.342162 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:50.342131    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dqwqh"]
Jul 28 11:36:50.348235 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:36:50.348204    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6364a4d1_df2f_46cd_9f17_335914a957c9.slice/crio-e3bb8904c961d00d96ce338ea42945d7eea7a093dc08e4cdeb6a1f2ed5c8e7f2 WatchSource:0}: Error finding container e3bb8904c961d00d96ce338ea42945d7eea7a093dc08e4cdeb6a1f2ed5c8e7f2: Status 404 returned error can't find the container with id e3bb8904c961d00d96ce338ea42945d7eea7a093dc08e4cdeb6a1f2ed5c8e7f2
Jul 28 11:36:50.437313 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:50.437273    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dqwqh" event={"ID":"6364a4d1-df2f-46cd-9f17-335914a957c9","Type":"ContainerStarted","Data":"e3bb8904c961d00d96ce338ea42945d7eea7a093dc08e4cdeb6a1f2ed5c8e7f2"}
Jul 28 11:36:51.443200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:51.443115    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dqwqh" event={"ID":"6364a4d1-df2f-46cd-9f17-335914a957c9","Type":"ContainerStarted","Data":"839fce7b2bfa3d193f43a1210b2baa4b5f629cfb87a0527e899feb0383aa2c3a"}
Jul 28 11:36:51.443200 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:51.443170    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dqwqh" event={"ID":"6364a4d1-df2f-46cd-9f17-335914a957c9","Type":"ContainerStarted","Data":"d0d232f26ae31993384c6f4bb251afac5f00828e74204181134d09ddedd0d6c6"}
Jul 28 11:36:51.459757 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:51.459708    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dqwqh" podStartSLOduration=374.663893717 podStartE2EDuration="6m15.459693978s" totalImagesPullingTime="795.800261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:36:51.458314582 +0000 UTC m=+375.915349066" watchObservedRunningTime="2026-07-28 11:36:51.459693978 +0000 UTC m=+375.916728435"
Jul 28 11:36:58.634393 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634354    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:36:58.634808 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634636    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="alertmanager" containerID="cri-o://20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f" gracePeriod=120
Jul 28 11:36:58.634808 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634708    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-metric" containerID="cri-o://8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08" gracePeriod=120
Jul 28 11:36:58.634808 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634737    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-web" containerID="cri-o://7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca" gracePeriod=120
Jul 28 11:36:58.635017 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634800    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="prom-label-proxy" containerID="cri-o://9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f" gracePeriod=120
Jul 28 11:36:58.635017 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634798    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy" containerID="cri-o://09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709" gracePeriod=120
Jul 28 11:36:58.635123 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:58.634985    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="config-reloader" containerID="cri-o://5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79" gracePeriod=120
Jul 28 11:36:59.478837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478804    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f" exitCode=0
Jul 28 11:36:59.478837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478827    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08" exitCode=0
Jul 28 11:36:59.478837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478834    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709" exitCode=0
Jul 28 11:36:59.478837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478839    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79" exitCode=0
Jul 28 11:36:59.478837 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478846    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f" exitCode=0
Jul 28 11:36:59.479154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478878    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"}
Jul 28 11:36:59.479154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478959    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"}
Jul 28 11:36:59.479154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.478986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"}
Jul 28 11:36:59.479154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.479010    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"}
Jul 28 11:36:59.479154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.479033    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"}
Jul 28 11:36:59.874247 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.874220    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:36:59.888777 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.888753    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.888918 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.888809    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.888918 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.888833    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.888918 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.888862    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.888918 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.888886    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889150 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889019    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889150 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889055    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889150 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889081    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889150 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889113    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889152    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889182    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889194    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889208    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889219    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mbtx\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889244    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets\") pod \"729c0887-ad3a-4938-ba45-dd4ad7f55917\" (UID: \"729c0887-ad3a-4938-ba45-dd4ad7f55917\") "
Jul 28 11:36:59.889353 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889267    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:36:59.889727 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889453    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-metrics-client-ca\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.889727 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889474    2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.889727 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.889490    2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-alertmanager-main-db\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.892159 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892055    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:59.892159 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892122    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.892506 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892463    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.892585 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892530    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.892693 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892651    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.892950 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.892922    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx" (OuterVolumeSpecName: "kube-api-access-7mbtx") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "kube-api-access-7mbtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:59.893946 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.893919    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out" (OuterVolumeSpecName: "config-out") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:59.894176 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.894153    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume" (OuterVolumeSpecName: "config-volume") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.896975 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.896855    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.904152 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.904130    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config" (OuterVolumeSpecName: "web-config") pod "729c0887-ad3a-4938-ba45-dd4ad7f55917" (UID: "729c0887-ad3a-4938-ba45-dd4ad7f55917"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:59.989833 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989802    2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.989833 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989830    2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.989833 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989840    2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-out\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989849    2552 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-config-volume\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989857    2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-web-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989865    2552 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-cluster-tls-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989873    2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-main-tls\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989882    2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/729c0887-ad3a-4938-ba45-dd4ad7f55917-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989890    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mbtx\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-kube-api-access-7mbtx\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:36:59.990057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:36:59.989917    2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/729c0887-ad3a-4938-ba45-dd4ad7f55917-tls-assets\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:37:00.488262 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.488228    2552 generic.go:356] "Generic (PLEG): container finished" podID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerID="7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca" exitCode=0
Jul 28 11:37:00.488452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.488274    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"}
Jul 28 11:37:00.488452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.488319    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.488452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.488337    2552 scope.go:117] "RemoveContainer" containerID="9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"
Jul 28 11:37:00.488452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.488327    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"729c0887-ad3a-4938-ba45-dd4ad7f55917","Type":"ContainerDied","Data":"a242fa827e5e47093ee0558d9143a6f50d9638ca14f048a9239f5df98afffe70"}
Jul 28 11:37:00.496117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.496098    2552 scope.go:117] "RemoveContainer" containerID="8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"
Jul 28 11:37:00.502869 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.502848    2552 scope.go:117] "RemoveContainer" containerID="09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"
Jul 28 11:37:00.509293 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.509276    2552 scope.go:117] "RemoveContainer" containerID="7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"
Jul 28 11:37:00.512010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.511985    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:37:00.515334 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.515302    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:37:00.516512 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.516499    2552 scope.go:117] "RemoveContainer" containerID="5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"
Jul 28 11:37:00.523540 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.523526    2552 scope.go:117] "RemoveContainer" containerID="20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"
Jul 28 11:37:00.530280 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.530265    2552 scope.go:117] "RemoveContainer" containerID="402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87"
Jul 28 11:37:00.536821 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.536803    2552 scope.go:117] "RemoveContainer" containerID="9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"
Jul 28 11:37:00.537099 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.537077    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f\": container with ID starting with 9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f not found: ID does not exist" containerID="9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"
Jul 28 11:37:00.537180 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537109    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f"} err="failed to get container status \"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f\": rpc error: code = NotFound desc = could not find container \"9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f\": container with ID starting with 9322483cf7fc21e58a8d4462d17897bb7c42559ad9d3a4124265dd261b85b24f not found: ID does not exist"
Jul 28 11:37:00.537180 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537134    2552 scope.go:117] "RemoveContainer" containerID="8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"
Jul 28 11:37:00.537364 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.537346    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08\": container with ID starting with 8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08 not found: ID does not exist" containerID="8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"
Jul 28 11:37:00.537405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537369    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08"} err="failed to get container status \"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08\": rpc error: code = NotFound desc = could not find container \"8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08\": container with ID starting with 8570bed466ae5e9cc7692124446cc884cc794f1a4581e9f0a4ccc10675d7db08 not found: ID does not exist"
Jul 28 11:37:00.537405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537385    2552 scope.go:117] "RemoveContainer" containerID="09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"
Jul 28 11:37:00.537587 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.537568    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709\": container with ID starting with 09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709 not found: ID does not exist" containerID="09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"
Jul 28 11:37:00.537624 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537590    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709"} err="failed to get container status \"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709\": rpc error: code = NotFound desc = could not find container \"09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709\": container with ID starting with 09909a5744ae553159485305cada86861d3577ac699ea24c28c7ee5bbc938709 not found: ID does not exist"
Jul 28 11:37:00.537624 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537601    2552 scope.go:117] "RemoveContainer" containerID="7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"
Jul 28 11:37:00.537825 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.537809    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca\": container with ID starting with 7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca not found: ID does not exist" containerID="7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"
Jul 28 11:37:00.537888 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537829    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca"} err="failed to get container status \"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca\": rpc error: code = NotFound desc = could not find container \"7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca\": container with ID starting with 7ea95c5e0d3dcc963e68cb5c7f253894accfe8d8209f2c8e2ec0020330bea5ca not found: ID does not exist"
Jul 28 11:37:00.537888 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.537840    2552 scope.go:117] "RemoveContainer" containerID="5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"
Jul 28 11:37:00.538069 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.538052    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79\": container with ID starting with 5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79 not found: ID does not exist" containerID="5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"
Jul 28 11:37:00.538107 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.538074    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79"} err="failed to get container status \"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79\": rpc error: code = NotFound desc = could not find container \"5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79\": container with ID starting with 5770d8c46c931ac40bdbb8539b6f37d5b09db1b9049e356db24af765badc0c79 not found: ID does not exist"
Jul 28 11:37:00.538107 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.538086    2552 scope.go:117] "RemoveContainer" containerID="20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"
Jul 28 11:37:00.538321 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.538302    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f\": container with ID starting with 20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f not found: ID does not exist" containerID="20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"
Jul 28 11:37:00.538374 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.538325    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f"} err="failed to get container status \"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f\": rpc error: code = NotFound desc = could not find container \"20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f\": container with ID starting with 20ff5b7f8d47e9cfe7eda348263e7f04f6ea212fc5a61420fbc680b958b3c94f not found: ID does not exist"
Jul 28 11:37:00.538374 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.538340    2552 scope.go:117] "RemoveContainer" containerID="402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87"
Jul 28 11:37:00.538567 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:37:00.538553    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87\": container with ID starting with 402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87 not found: ID does not exist" containerID="402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87"
Jul 28 11:37:00.538603 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.538571    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87"} err="failed to get container status \"402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87\": rpc error: code = NotFound desc = could not find container \"402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87\": container with ID starting with 402144dd04ae39292abcdaaf073661ce3ab30d110942412cec0dce61fe453e87 not found: ID does not exist"
Jul 28 11:37:00.545117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545097    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:37:00.545381 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545367    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" containerName="console"
Jul 28 11:37:00.545421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545384    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" containerName="console"
Jul 28 11:37:00.545421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545395    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-metric"
Jul 28 11:37:00.545421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545401    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-metric"
Jul 28 11:37:00.545421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545409    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-web"
Jul 28 11:37:00.545421 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545414    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-web"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545423    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="init-config-reloader"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545428    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="init-config-reloader"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545436    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="alertmanager"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545441    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="alertmanager"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545447    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerName="registry"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545452    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerName="registry"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545462    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="config-reloader"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545467    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="config-reloader"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545473    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545478    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545485    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="prom-label-proxy"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545490    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="prom-label-proxy"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545534    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e66f8c-85eb-47b4-894d-9f1e172bfb32" containerName="console"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545542    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="config-reloader"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545547    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="prom-label-proxy"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545554    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-web"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545559    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy-metric"
Jul 28 11:37:00.545563 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545566    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="kube-rbac-proxy"
Jul 28 11:37:00.546118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545572    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" containerName="alertmanager"
Jul 28 11:37:00.546118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.545576    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="891bad0a-f477-4b26-91bb-42fa6104f86e" containerName="registry"
Jul 28 11:37:00.550346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.550331    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.552335 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552283    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 11:37:00.552446 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552398    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 11:37:00.552532 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552441    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 11:37:00.552532 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552443    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ggs5\""
Jul 28 11:37:00.552532 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552495    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 11:37:00.552819 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552802    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 11:37:00.552906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552832    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 11:37:00.552906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552804    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 11:37:00.552906 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.552875    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 11:37:00.557996 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.557974    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 11:37:00.563720 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.563699    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:37:00.594678 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594649    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594678 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594682    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mclqr\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-kube-api-access-mclqr\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594824 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594700    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-config-volume\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594824 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594718    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-config-out\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594824 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594791    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594824 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594820    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594867    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594888    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594937    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594960    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.594987 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594976    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-web-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.595129 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.594999    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.595129 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.595044    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696051 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696016    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696051 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696053    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696240 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696080    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696240 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696201    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696238    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-web-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696273    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696304    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696341    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696377    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mclqr\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-kube-api-access-mclqr\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696393    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696404    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-config-volume\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696455    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-config-out\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696497    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696528 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696521    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.696815 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.696802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.697223 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.697197    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2428693a-1121-4121-acd8-6ff5714b0566-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699167    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-web-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699303 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699210    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2428693a-1121-4121-acd8-6ff5714b0566-config-out\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699303 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699239    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699403 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699340    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-config-volume\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699403 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699351    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699540 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699509    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699719 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699701    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.699784 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.699768    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2428693a-1121-4121-acd8-6ff5714b0566-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.701021 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.701006    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.706219 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.706202    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mclqr\" (UniqueName: \"kubernetes.io/projected/2428693a-1121-4121-acd8-6ff5714b0566-kube-api-access-mclqr\") pod \"alertmanager-main-0\" (UID: \"2428693a-1121-4121-acd8-6ff5714b0566\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.859229 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.859140    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 11:37:00.996155 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:00.996117    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 11:37:00.998663 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:37:00.998639    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2428693a_1121_4121_acd8_6ff5714b0566.slice/crio-c866ad62d75d4720121552e3e4eea7f44cb85bcd476e2764a5e4e953a706d407 WatchSource:0}: Error finding container c866ad62d75d4720121552e3e4eea7f44cb85bcd476e2764a5e4e953a706d407: Status 404 returned error can't find the container with id c866ad62d75d4720121552e3e4eea7f44cb85bcd476e2764a5e4e953a706d407
Jul 28 11:37:01.493804 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:01.493771    2552 generic.go:356] "Generic (PLEG): container finished" podID="2428693a-1121-4121-acd8-6ff5714b0566" containerID="efccf45f28e0d662f38bf03e0d9318b7fd9949f1346e5a0ac796266e002e4550" exitCode=0
Jul 28 11:37:01.493975 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:01.493826    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerDied","Data":"efccf45f28e0d662f38bf03e0d9318b7fd9949f1346e5a0ac796266e002e4550"}
Jul 28 11:37:01.493975 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:01.493873    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"c866ad62d75d4720121552e3e4eea7f44cb85bcd476e2764a5e4e953a706d407"}
Jul 28 11:37:02.108021 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.107926    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="729c0887-ad3a-4938-ba45-dd4ad7f55917" path="/var/lib/kubelet/pods/729c0887-ad3a-4938-ba45-dd4ad7f55917/volumes"
Jul 28 11:37:02.501461 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501424    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"8877ac4484ba554037e9e7eee67d3cafe35ec8f6fc9348506fed9b2239f0a9e5"}
Jul 28 11:37:02.501645 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501508    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"d8aec6b1f8ae86a746b06dfe8257a9be75732604a19c704153f11b7866801b36"}
Jul 28 11:37:02.501645 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501553    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"3769a0de7906c85fba9019b3b0a3674b0e3a3e964c2585a6c7b97e3e0af4b60b"}
Jul 28 11:37:02.501645 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501588    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"cfc4a341a1055311f59abb72d258d97016106e4df6cdbe8f48ddd21482304241"}
Jul 28 11:37:02.501645 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501627    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"e69ffb679caa08a5bcb389b5bf3eadcc3bf7922b3ed6bad80596463632dc9a60"}
Jul 28 11:37:02.501804 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.501710    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2428693a-1121-4121-acd8-6ff5714b0566","Type":"ContainerStarted","Data":"8e5fdaa6a0c23b38be6dafe1c9eec6a6ae68f286b66a2e7a6b958d921133bb28"}
Jul 28 11:37:02.526387 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.526333    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.509228429 podStartE2EDuration="2.526316325s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.087896ms" isStatefulPod=false podCreationTimestamp="2026-07-28 11:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:37:02.524571574 +0000 UTC m=+386.981606031" watchObservedRunningTime="2026-07-28 11:37:02.526316325 +0000 UTC m=+386.983350782"
Jul 28 11:37:02.664875 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.664839    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"]
Jul 28 11:37:02.669342 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.669327    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.671303 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.671283    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 11:37:02.671544 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.671531    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 11:37:02.671598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.671573    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 11:37:02.672117 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.672098    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 11:37:02.672189 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.672121    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 11:37:02.672371 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.672353    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2f8td\""
Jul 28 11:37:02.677967 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.677937    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 11:37:02.680802 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.680779    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"]
Jul 28 11:37:02.711475 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711446    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-federate-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711475 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711477    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711499    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-metrics-client-ca\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711564    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711587    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x8j8\" (UniqueName: \"kubernetes.io/projected/da3246e1-4a3f-4d85-a60a-d6d210f29068-kube-api-access-5x8j8\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711625 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711618    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711747 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711639    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.711747 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.711669    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-serving-certs-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812413 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812344    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-metrics-client-ca\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812413 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812388    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812413 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812410    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x8j8\" (UniqueName: \"kubernetes.io/projected/da3246e1-4a3f-4d85-a60a-d6d210f29068-kube-api-access-5x8j8\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812435    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812457    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812480    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-serving-certs-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812506    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-federate-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.812618 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.812525    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.813263 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.813227    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-metrics-client-ca\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.813377 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.813356    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-serving-certs-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.813480 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.813451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.815027 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.815005    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.815198 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.815179    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-federate-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.815276 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.815260    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-secret-telemeter-client\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.815321 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.815305    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da3246e1-4a3f-4d85-a60a-d6d210f29068-telemeter-client-tls\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.820700 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.820681    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x8j8\" (UniqueName: \"kubernetes.io/projected/da3246e1-4a3f-4d85-a60a-d6d210f29068-kube-api-access-5x8j8\") pod \"telemeter-client-6dd9bfb7d8-kv8mr\" (UID: \"da3246e1-4a3f-4d85-a60a-d6d210f29068\") " pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:02.979495 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:02.979454    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"
Jul 28 11:37:03.108780 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:03.108751    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr"]
Jul 28 11:37:03.110965 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:37:03.110933    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3246e1_4a3f_4d85_a60a_d6d210f29068.slice/crio-0dcb0771ee6033691f53bfec08e252bb916b347c74b3c66fddb2fd2e689ba9cc WatchSource:0}: Error finding container 0dcb0771ee6033691f53bfec08e252bb916b347c74b3c66fddb2fd2e689ba9cc: Status 404 returned error can't find the container with id 0dcb0771ee6033691f53bfec08e252bb916b347c74b3c66fddb2fd2e689ba9cc
Jul 28 11:37:03.506847 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:03.506805    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr" event={"ID":"da3246e1-4a3f-4d85-a60a-d6d210f29068","Type":"ContainerStarted","Data":"0dcb0771ee6033691f53bfec08e252bb916b347c74b3c66fddb2fd2e689ba9cc"}
Jul 28 11:37:05.516797 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:05.516766    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr" event={"ID":"da3246e1-4a3f-4d85-a60a-d6d210f29068","Type":"ContainerStarted","Data":"1d4635d3a6c42e91829b4cdcb6418939d47ab66505fb42eac8e5794f40f727c3"}
Jul 28 11:37:05.517199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:05.516827    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr" event={"ID":"da3246e1-4a3f-4d85-a60a-d6d210f29068","Type":"ContainerStarted","Data":"822dfb48329ddd8df8b4405e21fe05ec6f1cda90abf0ba2a7e60121c4d766756"}
Jul 28 11:37:05.517199 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:05.516848    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr" event={"ID":"da3246e1-4a3f-4d85-a60a-d6d210f29068","Type":"ContainerStarted","Data":"16dddced22b99f0876ffdd027f8d5f762ba32b269e8e6315c5d7432bce00e733"}
Jul 28 11:37:05.541558 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:05.541514    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6dd9bfb7d8-kv8mr" podStartSLOduration=1.682029502 podStartE2EDuration="3.541498512s" totalImagesPullingTime="1.85946901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:37:05.539925838 +0000 UTC m=+389.996960296" watchObservedRunningTime="2026-07-28 11:37:05.541498512 +0000 UTC m=+389.998532970"
Jul 28 11:37:06.275536 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.275500    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:37:06.278958 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.278936    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.281677 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281652    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281685    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281695    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281744    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281764    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281786    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 11:37:06.281826 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.281710    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dzxgt\""
Jul 28 11:37:06.282141 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.282121    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 11:37:06.290605 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.290582    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 11:37:06.297969 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.297942    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:37:06.346762 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346737    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.346923 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346774    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.346923 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.346923 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346811    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgggj\" (UniqueName: \"kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.346923 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346855    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.347130 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.346939    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.347130 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.347036    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.449815 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.449775    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.449815 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.449819    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.449848    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.449869    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.449970    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgggj\" (UniqueName: \"kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450025    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450118 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450055    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450561 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450538    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450765 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450742    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450852 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450821    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.450973 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.450876    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.452234 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.452211    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.452505 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.452488    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.462063 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.462042    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgggj\" (UniqueName: \"kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj\") pod \"console-7766bc68f6-rrpd2\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") " pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.589010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.588941    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:06.740607 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:06.740571    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:37:06.744341 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:37:06.744315    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098b3612_94da_462a_aeb5_a3b5857ed035.slice/crio-5ba4e2a95b18f0f9062730f1a428f166b0e9a8a0961ac639820f82b802914c50 WatchSource:0}: Error finding container 5ba4e2a95b18f0f9062730f1a428f166b0e9a8a0961ac639820f82b802914c50: Status 404 returned error can't find the container with id 5ba4e2a95b18f0f9062730f1a428f166b0e9a8a0961ac639820f82b802914c50
Jul 28 11:37:07.525981 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:07.525942    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766bc68f6-rrpd2" event={"ID":"098b3612-94da-462a-aeb5-a3b5857ed035","Type":"ContainerStarted","Data":"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"}
Jul 28 11:37:07.526154 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:07.525995    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766bc68f6-rrpd2" event={"ID":"098b3612-94da-462a-aeb5-a3b5857ed035","Type":"ContainerStarted","Data":"5ba4e2a95b18f0f9062730f1a428f166b0e9a8a0961ac639820f82b802914c50"}
Jul 28 11:37:07.545185 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:07.545139    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7766bc68f6-rrpd2" podStartSLOduration=1.545125202 podStartE2EDuration="1.545125202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:37:07.543981999 +0000 UTC m=+392.001016457" watchObservedRunningTime="2026-07-28 11:37:07.545125202 +0000 UTC m=+392.002159659"
Jul 28 11:37:16.589158 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:16.589120    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:16.589562 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:16.589200    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:16.593854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:16.593829    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:37:17.573753 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:37:17.573722    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:38:16.312121 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.312090    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:38:16.315768 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.315745    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.325657 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.325631    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:38:16.419931 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.419881    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420096 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.419936    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420096 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.419980    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjcq\" (UniqueName: \"kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420096 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.420007    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420206 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.420095    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420206 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.420151    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.420206 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.420167    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.520813 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520777    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520823    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520839    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520862    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.520954    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjcq\" (UniqueName: \"kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.521006    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521765 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.521736    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521889 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.521866    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.521965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.521918    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.522227 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.522209    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.523787 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.523762    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.523963 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.523943    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.529928 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.529885    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjcq\" (UniqueName: \"kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq\") pod \"console-bdbb4b886-z4jr9\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") " pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.625972 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.625881    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:16.755715 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.755686    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:38:16.759250 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:38:16.759223    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9710d061_8d1d_4468_993f_7901a182a437.slice/crio-328a3bf2c4d7187bf55169a272c27b92b11c3e6572de0037b1fbb731ad6d67e6 WatchSource:0}: Error finding container 328a3bf2c4d7187bf55169a272c27b92b11c3e6572de0037b1fbb731ad6d67e6: Status 404 returned error can't find the container with id 328a3bf2c4d7187bf55169a272c27b92b11c3e6572de0037b1fbb731ad6d67e6
Jul 28 11:38:16.810699 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:16.810672    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdbb4b886-z4jr9" event={"ID":"9710d061-8d1d-4468-993f-7901a182a437","Type":"ContainerStarted","Data":"328a3bf2c4d7187bf55169a272c27b92b11c3e6572de0037b1fbb731ad6d67e6"}
Jul 28 11:38:17.815692 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:17.815650    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdbb4b886-z4jr9" event={"ID":"9710d061-8d1d-4468-993f-7901a182a437","Type":"ContainerStarted","Data":"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"}
Jul 28 11:38:17.832167 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:17.832110    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bdbb4b886-z4jr9" podStartSLOduration=1.832091167 podStartE2EDuration="1.832091167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:38:17.831748438 +0000 UTC m=+462.288782896" watchObservedRunningTime="2026-07-28 11:38:17.832091167 +0000 UTC m=+462.289125625"
Jul 28 11:38:26.626466 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:26.626424    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:26.626858 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:26.626488    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:26.631096 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:26.631073    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:26.855610 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:26.855580    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:38:26.904004 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:26.903922    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:38:51.922780 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:51.922710    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7766bc68f6-rrpd2" podUID="098b3612-94da-462a-aeb5-a3b5857ed035" containerName="console" containerID="cri-o://1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f" gracePeriod=15
Jul 28 11:38:52.162545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.162520    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:38:52.216293 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216266    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgggj\" (UniqueName: \"kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216312    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216337    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216361    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216398    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216452    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216721 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216487    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert\") pod \"098b3612-94da-462a-aeb5-a3b5857ed035\" (UID: \"098b3612-94da-462a-aeb5-a3b5857ed035\") "
Jul 28 11:38:52.216854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216811    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:38:52.216854 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216825    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config" (OuterVolumeSpecName: "console-config") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:38:52.216993 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216855    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca" (OuterVolumeSpecName: "service-ca") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:38:52.216993 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.216957    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:38:52.218975 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.218952    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:38:52.219072 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.219012    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj" (OuterVolumeSpecName: "kube-api-access-xgggj") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "kube-api-access-xgggj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:38:52.219072 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.219038    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "098b3612-94da-462a-aeb5-a3b5857ed035" (UID: "098b3612-94da-462a-aeb5-a3b5857ed035"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:38:52.317249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317217    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgggj\" (UniqueName: \"kubernetes.io/projected/098b3612-94da-462a-aeb5-a3b5857ed035-kube-api-access-xgggj\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317248    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-trusted-ca-bundle\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317249 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317259    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-console-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317474 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317267    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-service-ca\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317474 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317276    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-oauth-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317474 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317284    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/098b3612-94da-462a-aeb5-a3b5857ed035-oauth-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.317474 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.317292    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/098b3612-94da-462a-aeb5-a3b5857ed035-console-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:38:52.962174 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.962131    2552 generic.go:356] "Generic (PLEG): container finished" podID="098b3612-94da-462a-aeb5-a3b5857ed035" containerID="1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f" exitCode=0
Jul 28 11:38:52.962598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.962186    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766bc68f6-rrpd2" event={"ID":"098b3612-94da-462a-aeb5-a3b5857ed035","Type":"ContainerDied","Data":"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"}
Jul 28 11:38:52.962598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.962220    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7766bc68f6-rrpd2"
Jul 28 11:38:52.962598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.962282    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7766bc68f6-rrpd2" event={"ID":"098b3612-94da-462a-aeb5-a3b5857ed035","Type":"ContainerDied","Data":"5ba4e2a95b18f0f9062730f1a428f166b0e9a8a0961ac639820f82b802914c50"}
Jul 28 11:38:52.962598 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.962311    2552 scope.go:117] "RemoveContainer" containerID="1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"
Jul 28 11:38:52.971040 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.971022    2552 scope.go:117] "RemoveContainer" containerID="1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"
Jul 28 11:38:52.971303 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:38:52.971283    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f\": container with ID starting with 1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f not found: ID does not exist" containerID="1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"
Jul 28 11:38:52.971349 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.971311    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f"} err="failed to get container status \"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f\": rpc error: code = NotFound desc = could not find container \"1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f\": container with ID starting with 1509ac9ec6f208117b9be49632ae0f5be8303c60fed2694e7e15de4f7e3e7b2f not found: ID does not exist"
Jul 28 11:38:52.982080 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.982058    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:38:52.987423 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:52.987404    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7766bc68f6-rrpd2"]
Jul 28 11:38:54.108578 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:38:54.108542    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="098b3612-94da-462a-aeb5-a3b5857ed035" path="/var/lib/kubelet/pods/098b3612-94da-462a-aeb5-a3b5857ed035/volumes"
Jul 28 11:40:14.571193 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.571160    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcrvc"]
Jul 28 11:40:14.571697 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.571480    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="098b3612-94da-462a-aeb5-a3b5857ed035" containerName="console"
Jul 28 11:40:14.571697 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.571490    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="098b3612-94da-462a-aeb5-a3b5857ed035" containerName="console"
Jul 28 11:40:14.571697 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.571545    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="098b3612-94da-462a-aeb5-a3b5857ed035" containerName="console"
Jul 28 11:40:14.574513 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.574495    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.576606 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.576579    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-tgkzt\""
Jul 28 11:40:14.576723 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.576704    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 11:40:14.576993 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.576976    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 11:40:14.584347 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.584327    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcrvc"]
Jul 28 11:40:14.688947 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.688914    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wtd\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-kube-api-access-k9wtd\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.689108 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.688977    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.790162 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.790124    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wtd\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-kube-api-access-k9wtd\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.790317 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.790186    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.804925 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.804876    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wtd\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-kube-api-access-k9wtd\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.805393 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.805376    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e68121ab-dcaf-4da1-a0ca-8dd704c1277a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcrvc\" (UID: \"e68121ab-dcaf-4da1-a0ca-8dd704c1277a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:14.893265 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:14.893190    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:15.025870 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:15.025834    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcrvc"]
Jul 28 11:40:15.028831 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:40:15.028804    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode68121ab_dcaf_4da1_a0ca_8dd704c1277a.slice/crio-5f0f7a7d721598fc6b9d05c780c92fbf0f2539caaf574bcd5176a86cbe93109a WatchSource:0}: Error finding container 5f0f7a7d721598fc6b9d05c780c92fbf0f2539caaf574bcd5176a86cbe93109a: Status 404 returned error can't find the container with id 5f0f7a7d721598fc6b9d05c780c92fbf0f2539caaf574bcd5176a86cbe93109a
Jul 28 11:40:15.272961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:15.272923    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc" event={"ID":"e68121ab-dcaf-4da1-a0ca-8dd704c1277a","Type":"ContainerStarted","Data":"5f0f7a7d721598fc6b9d05c780c92fbf0f2539caaf574bcd5176a86cbe93109a"}
Jul 28 11:40:16.826437 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.826403    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"]
Jul 28 11:40:16.830746 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.830730    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:16.832448 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.832427    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z49qx\""
Jul 28 11:40:16.838206 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.838185    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"]
Jul 28 11:40:16.908162 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.908128    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkw8f\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-kube-api-access-vkw8f\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:16.908310 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:16.908183    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.008919 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.008863    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkw8f\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-kube-api-access-vkw8f\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.009089 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.008946    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.017538 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.017498    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.017655 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.017610    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkw8f\" (UniqueName: \"kubernetes.io/projected/c11b852a-5993-4658-8aa0-5278dc5203cb-kube-api-access-vkw8f\") pod \"cert-manager-cainjector-559cdd8c87-5phmg\" (UID: \"c11b852a-5993-4658-8aa0-5278dc5203cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.140486 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.140396    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"
Jul 28 11:40:17.308885 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:17.308644    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5phmg"]
Jul 28 11:40:17.312278 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:40:17.312249    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc11b852a_5993_4658_8aa0_5278dc5203cb.slice/crio-f6e413be36e435684d49c4ae4fc56132333637e15c9a1c2452df5455e73a8aaf WatchSource:0}: Error finding container f6e413be36e435684d49c4ae4fc56132333637e15c9a1c2452df5455e73a8aaf: Status 404 returned error can't find the container with id f6e413be36e435684d49c4ae4fc56132333637e15c9a1c2452df5455e73a8aaf
Jul 28 11:40:18.289057 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:18.289012    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg" event={"ID":"c11b852a-5993-4658-8aa0-5278dc5203cb","Type":"ContainerStarted","Data":"f6e413be36e435684d49c4ae4fc56132333637e15c9a1c2452df5455e73a8aaf"}
Jul 28 11:40:20.299794 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:20.299752    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc" event={"ID":"e68121ab-dcaf-4da1-a0ca-8dd704c1277a","Type":"ContainerStarted","Data":"5ee27ec5e7719f5d633a798eca6ddb667967c76f13f2992044e53fef16158a27"}
Jul 28 11:40:20.300268 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:20.299908    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:20.301145 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:20.301121    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg" event={"ID":"c11b852a-5993-4658-8aa0-5278dc5203cb","Type":"ContainerStarted","Data":"3a1f3cf0d51de2d392279b69a51e0ac67f53ec5d972b5f67ad5b21be14bfbe6d"}
Jul 28 11:40:20.314169 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:20.314125    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc" podStartSLOduration=1.877600246 podStartE2EDuration="6.314111144s" totalImagesPullingTime="4.436510898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:40:20.313385932 +0000 UTC m=+584.770420390" watchObservedRunningTime="2026-07-28 11:40:20.314111144 +0000 UTC m=+584.771145601"
Jul 28 11:40:20.327508 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:20.327467    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5phmg" podStartSLOduration=2.174774015 podStartE2EDuration="4.327455918s" totalImagesPullingTime="2.152681903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:40:20.326345266 +0000 UTC m=+584.783379723" watchObservedRunningTime="2026-07-28 11:40:20.327455918 +0000 UTC m=+584.784490378"
Jul 28 11:40:26.308010 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:26.307979    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bcrvc"
Jul 28 11:40:48.665078 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.665043    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"]
Jul 28 11:40:48.668643 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.668582    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.670857 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.670837    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 11:40:48.671282 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.671261    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 11:40:48.671359 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.671294    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 11:40:48.671359 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.671336    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sw542\""
Jul 28 11:40:48.671457 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.671410    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 11:40:48.683924 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.683887    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"]
Jul 28 11:40:48.789653 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.789617    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.789820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.789660    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxnkn\" (UniqueName: \"kubernetes.io/projected/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-kube-api-access-kxnkn\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.789820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.789693    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.789820 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.789763    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-manifests\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.789954 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.789838    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-tmp\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890265 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890227    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890438 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890274    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-manifests\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890438 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-tmp\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890438 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890342    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890438 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890364    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxnkn\" (UniqueName: \"kubernetes.io/projected/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-kube-api-access-kxnkn\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.890784 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.890761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-manifests\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.892521 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.892496    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-tmp\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.892816 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.892793    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.892889 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.892810    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.908189 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.908159    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxnkn\" (UniqueName: \"kubernetes.io/projected/624b4bc4-e9f3-447b-b96d-2a5e35dc4d60-kube-api-access-kxnkn\") pod \"opendatahub-operator-controller-manager-6755d7577-jfn6z\" (UID: \"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:48.979179 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:48.979134    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:49.119052 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:49.119004    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"]
Jul 28 11:40:49.121939 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:40:49.121912    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624b4bc4_e9f3_447b_b96d_2a5e35dc4d60.slice/crio-0b6a7fba35b73c355db31f9e0f3150f34021b31795e8a9732fcd3803dbd328c7 WatchSource:0}: Error finding container 0b6a7fba35b73c355db31f9e0f3150f34021b31795e8a9732fcd3803dbd328c7: Status 404 returned error can't find the container with id 0b6a7fba35b73c355db31f9e0f3150f34021b31795e8a9732fcd3803dbd328c7
Jul 28 11:40:49.123589 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:49.123571    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:40:49.425011 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:49.424923    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z" event={"ID":"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60","Type":"ContainerStarted","Data":"0b6a7fba35b73c355db31f9e0f3150f34021b31795e8a9732fcd3803dbd328c7"}
Jul 28 11:40:52.443433 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:52.443396    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z" event={"ID":"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60","Type":"ContainerStarted","Data":"fe50df34682555628d1516d588f09cf23fc0c936aff631ade45f3e3eb377c46a"}
Jul 28 11:40:53.449044 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:53.449008    2552 generic.go:356] "Generic (PLEG): container finished" podID="624b4bc4-e9f3-447b-b96d-2a5e35dc4d60" containerID="fe50df34682555628d1516d588f09cf23fc0c936aff631ade45f3e3eb377c46a" exitCode=0
Jul 28 11:40:53.449434 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:53.449080    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z" event={"ID":"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60","Type":"ContainerDied","Data":"fe50df34682555628d1516d588f09cf23fc0c936aff631ade45f3e3eb377c46a"}
Jul 28 11:40:54.456531 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:54.456487    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z" event={"ID":"624b4bc4-e9f3-447b-b96d-2a5e35dc4d60","Type":"ContainerStarted","Data":"315ac83cc363fc957ce012ebc7b5799330a6c74e6bfd67ae37af2b4ac521c546"}
Jul 28 11:40:54.457015 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:54.456609    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:40:54.479455 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:40:54.479401    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z" podStartSLOduration=1.977755385 podStartE2EDuration="6.479383125s" totalImagesPullingTime="3.485184128s" totalInitContainerRuntime="1.016443612s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:40:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:40:54.477836178 +0000 UTC m=+618.934870655" watchObservedRunningTime="2026-07-28 11:40:54.479383125 +0000 UTC m=+618.936417629"
Jul 28 11:41:05.464007 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:05.463921    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6755d7577-jfn6z"
Jul 28 11:41:07.857623 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.857592    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"]
Jul 28 11:41:07.861310 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.861286    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:07.863404 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.863380    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 11:41:07.863663 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.863648    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cmk55\""
Jul 28 11:41:07.863727 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.863648    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 11:41:07.863993 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.863968    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 11:41:07.864193 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.864178    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 11:41:07.872594 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.872563    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"]
Jul 28 11:41:07.962286 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.962254    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93386518-d4ed-4ceb-8706-e8daf0821ff9-tls-certs\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:07.962467 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.962310    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtv7\" (UniqueName: \"kubernetes.io/projected/93386518-d4ed-4ceb-8706-e8daf0821ff9-kube-api-access-dhtv7\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:07.962467 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:07.962348    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93386518-d4ed-4ceb-8706-e8daf0821ff9-tmp\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.063453 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.063409    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93386518-d4ed-4ceb-8706-e8daf0821ff9-tls-certs\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.063639 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.063478    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtv7\" (UniqueName: \"kubernetes.io/projected/93386518-d4ed-4ceb-8706-e8daf0821ff9-kube-api-access-dhtv7\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.063639 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.063568    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93386518-d4ed-4ceb-8706-e8daf0821ff9-tmp\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.065824 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.065801    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93386518-d4ed-4ceb-8706-e8daf0821ff9-tmp\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.065961 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.065922    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93386518-d4ed-4ceb-8706-e8daf0821ff9-tls-certs\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.082637 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.082596    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtv7\" (UniqueName: \"kubernetes.io/projected/93386518-d4ed-4ceb-8706-e8daf0821ff9-kube-api-access-dhtv7\") pod \"kube-auth-proxy-8489dc4db9-rbztv\" (UID: \"93386518-d4ed-4ceb-8706-e8daf0821ff9\") " pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.173431 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.173351    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"
Jul 28 11:41:08.321741 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.321709    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv"]
Jul 28 11:41:08.324579 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:41:08.324546    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93386518_d4ed_4ceb_8706_e8daf0821ff9.slice/crio-2feb42d3c8d35dd4f5064e15af0d3754220361faf75c7bbd02fb82c7c4f78949 WatchSource:0}: Error finding container 2feb42d3c8d35dd4f5064e15af0d3754220361faf75c7bbd02fb82c7c4f78949: Status 404 returned error can't find the container with id 2feb42d3c8d35dd4f5064e15af0d3754220361faf75c7bbd02fb82c7c4f78949
Jul 28 11:41:08.517274 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:08.517240    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv" event={"ID":"93386518-d4ed-4ceb-8706-e8daf0821ff9","Type":"ContainerStarted","Data":"2feb42d3c8d35dd4f5064e15af0d3754220361faf75c7bbd02fb82c7c4f78949"}
Jul 28 11:41:11.534222 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:11.534186    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv" event={"ID":"93386518-d4ed-4ceb-8706-e8daf0821ff9","Type":"ContainerStarted","Data":"4c8313e529e67b92f2b1dd59a61b9f62753e934e949246ad5a0fcfed5cb0406d"}
Jul 28 11:41:11.551310 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:11.551252    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8489dc4db9-rbztv" podStartSLOduration=1.742330471 podStartE2EDuration="4.551231003s" totalImagesPullingTime="2.808900532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:41:11.550271653 +0000 UTC m=+636.007306110" watchObservedRunningTime="2026-07-28 11:41:11.551231003 +0000 UTC m=+636.008265467"
Jul 28 11:41:15.569349 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.569311    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"]
Jul 28 11:41:15.572941 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.572913    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.575085 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.575063    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 11:41:15.575501 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.575480    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mvwwj\""
Jul 28 11:41:15.575744 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.575725    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 11:41:15.575952 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.575932    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 11:41:15.576062 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.576046    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 11:41:15.582649 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.582627    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"]
Jul 28 11:41:15.738393 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738354    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738399    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f95e161-5c34-4fce-915f-c37e57a514c9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738451    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738491    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738515    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738545 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738532    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.738750 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.738610    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrpxk\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-kube-api-access-nrpxk\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.847226 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.847130    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.847765 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.847589    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f95e161-5c34-4fce-915f-c37e57a514c9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.847934 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.847883    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.848025 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.848004    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.848195 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.848178    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.848330 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.848296    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.848469 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.848455    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrpxk\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-kube-api-access-nrpxk\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.848581 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.848503    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.850657 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.850631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.850760 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.850664    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f95e161-5c34-4fce-915f-c37e57a514c9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.851064 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.851040    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.851137 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.851039    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.858674 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.858653    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrpxk\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-kube-api-access-nrpxk\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.858757 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.858702    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f95e161-5c34-4fce-915f-c37e57a514c9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-v48hg\" (UID: \"6f95e161-5c34-4fce-915f-c37e57a514c9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:15.882755 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:15.882731    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:16.019639 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:16.019615    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"]
Jul 28 11:41:16.022302 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:41:16.022273    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f95e161_5c34_4fce_915f_c37e57a514c9.slice/crio-ce1284b9422e7b7b5adcd84f487740df189cde9977898c51fe73e370c68823de WatchSource:0}: Error finding container ce1284b9422e7b7b5adcd84f487740df189cde9977898c51fe73e370c68823de: Status 404 returned error can't find the container with id ce1284b9422e7b7b5adcd84f487740df189cde9977898c51fe73e370c68823de
Jul 28 11:41:16.559857 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:16.559823    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg" event={"ID":"6f95e161-5c34-4fce-915f-c37e57a514c9","Type":"ContainerStarted","Data":"ce1284b9422e7b7b5adcd84f487740df189cde9977898c51fe73e370c68823de"}
Jul 28 11:41:17.111519 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.111452    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"]
Jul 28 11:41:17.118350 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.118311    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.123868 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.123844    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 11:41:17.124078 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.123845    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 11:41:17.124195 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.124179    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cs4jt\""
Jul 28 11:41:17.124308 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.124215    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 11:41:17.124308 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.124213    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 11:41:17.124484 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.124470    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 11:41:17.138742 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.138714    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"]
Jul 28 11:41:17.159717 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.159616    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.159717 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.159677    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c696k\" (UniqueName: \"kubernetes.io/projected/5e2b9826-b0f5-4409-836d-c719892fc096-kube-api-access-c696k\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.159930 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.159725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e2b9826-b0f5-4409-836d-c719892fc096-manager-config\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.159930 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.159796    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-metrics-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.261148 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.261110    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.261339 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.261155    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c696k\" (UniqueName: \"kubernetes.io/projected/5e2b9826-b0f5-4409-836d-c719892fc096-kube-api-access-c696k\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.261339 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.261191    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e2b9826-b0f5-4409-836d-c719892fc096-manager-config\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.261339 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.261237    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-metrics-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.262039 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.262014    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e2b9826-b0f5-4409-836d-c719892fc096-manager-config\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.264035 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.264012    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.264213 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.264194    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e2b9826-b0f5-4409-836d-c719892fc096-metrics-cert\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.271979 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.271951    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c696k\" (UniqueName: \"kubernetes.io/projected/5e2b9826-b0f5-4409-836d-c719892fc096-kube-api-access-c696k\") pod \"lws-controller-manager-65bf6bb88c-thsnz\" (UID: \"5e2b9826-b0f5-4409-836d-c719892fc096\") " pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.430595 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.430503    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:17.582356 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:17.582334    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"]
Jul 28 11:41:17.585373 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:41:17.585339    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2b9826_b0f5_4409_836d_c719892fc096.slice/crio-cffbed304586a0ad19c0f4abf2c763856a66d86f9eed0885ef2052ee95343ea3 WatchSource:0}: Error finding container cffbed304586a0ad19c0f4abf2c763856a66d86f9eed0885ef2052ee95343ea3: Status 404 returned error can't find the container with id cffbed304586a0ad19c0f4abf2c763856a66d86f9eed0885ef2052ee95343ea3
Jul 28 11:41:18.570542 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:18.570509    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz" event={"ID":"5e2b9826-b0f5-4409-836d-c719892fc096","Type":"ContainerStarted","Data":"cffbed304586a0ad19c0f4abf2c763856a66d86f9eed0885ef2052ee95343ea3"}
Jul 28 11:41:18.651883 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:18.651811    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 11:41:18.652328 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:18.651973    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 11:41:19.579140 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:19.579100    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg" event={"ID":"6f95e161-5c34-4fce-915f-c37e57a514c9","Type":"ContainerStarted","Data":"04cfd064c4f47835bb95f991970ade574de6da3af92e1f4e518eafec16f17fac"}
Jul 28 11:41:19.579653 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:19.579265    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:19.581110 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:19.581073    2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-v48hg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 11:41:19.581237 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:19.581127    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg" podUID="6f95e161-5c34-4fce-915f-c37e57a514c9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 11:41:19.630130 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:19.630066    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg" podStartSLOduration=2.002923041 podStartE2EDuration="4.630045149s" totalImagesPullingTime="2.627122108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:41:19.627487033 +0000 UTC m=+644.084521491" watchObservedRunningTime="2026-07-28 11:41:19.630045149 +0000 UTC m=+644.087079607"
Jul 28 11:41:20.585431 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:20.585398    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-v48hg"
Jul 28 11:41:21.590190 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:21.590145    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz" event={"ID":"5e2b9826-b0f5-4409-836d-c719892fc096","Type":"ContainerStarted","Data":"ee7bd69692450bdf6b091bad3a566897187f5610e48a88ba33b55918788d7b44"}
Jul 28 11:41:21.590570 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:21.590429    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:41:21.612282 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:21.612232    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz" podStartSLOduration=1.457088295 podStartE2EDuration="4.612218177s" totalImagesPullingTime="3.155129882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:41:21.610839384 +0000 UTC m=+646.067873847" watchObservedRunningTime="2026-07-28 11:41:21.612218177 +0000 UTC m=+646.069252634"
Jul 28 11:41:32.596705 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:41:32.596673    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65bf6bb88c-thsnz"
Jul 28 11:42:24.029912 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:24.029866    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:42:49.055632 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.055543    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bdbb4b886-z4jr9" podUID="9710d061-8d1d-4468-993f-7901a182a437" containerName="console" containerID="cri-o://4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c" gracePeriod=15
Jul 28 11:42:49.302080 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.302048    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:42:49.374186 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374101    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374186 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374157    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374186 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374182    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374458 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374208    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qjcq\" (UniqueName: \"kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374458 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374223    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374458 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374257    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374458 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374306    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config\") pod \"9710d061-8d1d-4468-993f-7901a182a437\" (UID: \"9710d061-8d1d-4468-993f-7901a182a437\") "
Jul 28 11:42:49.374710 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374677    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca" (OuterVolumeSpecName: "service-ca") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:42:49.374770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374690    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:42:49.374770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374730    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:42:49.374770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374741    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config" (OuterVolumeSpecName: "console-config") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 11:42:49.374990 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374971    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-trusted-ca-bundle\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.375056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.374997    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-service-ca\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.375056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.375012    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-oauth-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.375056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.375026    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9710d061-8d1d-4468-993f-7901a182a437-console-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.376338 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.376314    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:42:49.376437 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.376417    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:42:49.376494 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.376461    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq" (OuterVolumeSpecName: "kube-api-access-4qjcq") pod "9710d061-8d1d-4468-993f-7901a182a437" (UID: "9710d061-8d1d-4468-993f-7901a182a437"). InnerVolumeSpecName "kube-api-access-4qjcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:42:49.475691 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.475655    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qjcq\" (UniqueName: \"kubernetes.io/projected/9710d061-8d1d-4468-993f-7901a182a437-kube-api-access-4qjcq\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.475691 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.475688    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-serving-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.475691 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.475698    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9710d061-8d1d-4468-993f-7901a182a437-console-oauth-config\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:42:49.985198 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.985157    2552 generic.go:356] "Generic (PLEG): container finished" podID="9710d061-8d1d-4468-993f-7901a182a437" containerID="4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c" exitCode=0
Jul 28 11:42:49.985440 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.985215    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdbb4b886-z4jr9"
Jul 28 11:42:49.985440 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.985248    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdbb4b886-z4jr9" event={"ID":"9710d061-8d1d-4468-993f-7901a182a437","Type":"ContainerDied","Data":"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"}
Jul 28 11:42:49.985440 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.985311    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdbb4b886-z4jr9" event={"ID":"9710d061-8d1d-4468-993f-7901a182a437","Type":"ContainerDied","Data":"328a3bf2c4d7187bf55169a272c27b92b11c3e6572de0037b1fbb731ad6d67e6"}
Jul 28 11:42:49.985440 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.985334    2552 scope.go:117] "RemoveContainer" containerID="4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"
Jul 28 11:42:49.994669 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.994644    2552 scope.go:117] "RemoveContainer" containerID="4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"
Jul 28 11:42:49.994999 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:42:49.994978    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c\": container with ID starting with 4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c not found: ID does not exist" containerID="4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"
Jul 28 11:42:49.995077 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:49.995011    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c"} err="failed to get container status \"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c\": rpc error: code = NotFound desc = could not find container \"4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c\": container with ID starting with 4eff2284d94efa3d3369bbee6552a369aa03994e2fbab10c19b9ecd2e556262c not found: ID does not exist"
Jul 28 11:42:50.007253 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:50.007228    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:42:50.013791 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:50.013767    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bdbb4b886-z4jr9"]
Jul 28 11:42:50.108532 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:42:50.108501    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9710d061-8d1d-4468-993f-7901a182a437" path="/var/lib/kubelet/pods/9710d061-8d1d-4468-993f-7901a182a437/volumes"
Jul 28 11:49:47.227138 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.227105    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:49:47.227551 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.227440    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9710d061-8d1d-4468-993f-7901a182a437" containerName="console"
Jul 28 11:49:47.227551 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.227451    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9710d061-8d1d-4468-993f-7901a182a437" containerName="console"
Jul 28 11:49:47.227551 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.227504    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9710d061-8d1d-4468-993f-7901a182a437" containerName="console"
Jul 28 11:49:47.230417 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.230401    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.233423 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.233399    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 11:49:47.233582 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.233399    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 11:49:47.233582 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.233400    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nprmb\""
Jul 28 11:49:47.243452 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.243429    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:49:47.361039 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.360990    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.361211 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.361130    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jp2t\" (UniqueName: \"kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.462255 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.462212    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jp2t\" (UniqueName: \"kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.462405 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.462272    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.464603 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.464580    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.470828 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.470804    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jp2t\" (UniqueName: \"kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t\") pod \"maas-controller-6954b9867f-m6sh8\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") " pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.540254 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.540174    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:47.669329 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.669301    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:49:47.671879 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:49:47.671853    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a151d1_16e9_466d_b684_c5f2100bd856.slice/crio-94f3f06519411e5fb9c6f63c7dc1b0e1baabc8c7513990d5161a0aa1e9df9a4d WatchSource:0}: Error finding container 94f3f06519411e5fb9c6f63c7dc1b0e1baabc8c7513990d5161a0aa1e9df9a4d: Status 404 returned error can't find the container with id 94f3f06519411e5fb9c6f63c7dc1b0e1baabc8c7513990d5161a0aa1e9df9a4d
Jul 28 11:49:47.673561 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.673537    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:49:47.823346 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:47.823265    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-m6sh8" event={"ID":"84a151d1-16e9-466d-b684-c5f2100bd856","Type":"ContainerStarted","Data":"94f3f06519411e5fb9c6f63c7dc1b0e1baabc8c7513990d5161a0aa1e9df9a4d"}
Jul 28 11:49:50.839438 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:50.839392    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-m6sh8" event={"ID":"84a151d1-16e9-466d-b684-c5f2100bd856","Type":"ContainerStarted","Data":"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"}
Jul 28 11:49:50.839992 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:50.839501    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:49:50.857591 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:49:50.857550    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6954b9867f-m6sh8" podStartSLOduration=1.604908212 podStartE2EDuration="3.857537565s" totalImagesPullingTime="2.252629353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:49:50.855877274 +0000 UTC m=+1155.312911732" watchObservedRunningTime="2026-07-28 11:49:50.857537565 +0000 UTC m=+1155.314572021"
Jul 28 11:50:01.847030 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:01.846938    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:50:06.495157 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.495117    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:50:06.495577 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.495324    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6954b9867f-m6sh8" podUID="84a151d1-16e9-466d-b684-c5f2100bd856" containerName="manager" containerID="cri-o://13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d" gracePeriod=30
Jul 28 11:50:06.839926 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.839887    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:50:06.921255 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.921216    2552 generic.go:356] "Generic (PLEG): container finished" podID="84a151d1-16e9-466d-b684-c5f2100bd856" containerID="13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d" exitCode=0
Jul 28 11:50:06.921415 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.921266    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-m6sh8"
Jul 28 11:50:06.921415 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.921302    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-m6sh8" event={"ID":"84a151d1-16e9-466d-b684-c5f2100bd856","Type":"ContainerDied","Data":"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"}
Jul 28 11:50:06.921415 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.921357    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-m6sh8" event={"ID":"84a151d1-16e9-466d-b684-c5f2100bd856","Type":"ContainerDied","Data":"94f3f06519411e5fb9c6f63c7dc1b0e1baabc8c7513990d5161a0aa1e9df9a4d"}
Jul 28 11:50:06.921415 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.921385    2552 scope.go:117] "RemoveContainer" containerID="13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"
Jul 28 11:50:06.929570 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.929553    2552 scope.go:117] "RemoveContainer" containerID="13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"
Jul 28 11:50:06.929836 ip-10-0-130-140 kubenswrapper[2552]: E0728 11:50:06.929817    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d\": container with ID starting with 13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d not found: ID does not exist" containerID="13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"
Jul 28 11:50:06.929919 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.929845    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d"} err="failed to get container status \"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d\": rpc error: code = NotFound desc = could not find container \"13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d\": container with ID starting with 13ecde93173f31c736c367a2357fcef2e835a9117a3cd3b5f15e3058bb0ff19d not found: ID does not exist"
Jul 28 11:50:06.941148 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.941128    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert\") pod \"84a151d1-16e9-466d-b684-c5f2100bd856\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") "
Jul 28 11:50:06.941220 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.941205    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jp2t\" (UniqueName: \"kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t\") pod \"84a151d1-16e9-466d-b684-c5f2100bd856\" (UID: \"84a151d1-16e9-466d-b684-c5f2100bd856\") "
Jul 28 11:50:06.943256 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.943229    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t" (OuterVolumeSpecName: "kube-api-access-4jp2t") pod "84a151d1-16e9-466d-b684-c5f2100bd856" (UID: "84a151d1-16e9-466d-b684-c5f2100bd856"). InnerVolumeSpecName "kube-api-access-4jp2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:50:06.943324 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:06.943259    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "84a151d1-16e9-466d-b684-c5f2100bd856" (UID: "84a151d1-16e9-466d-b684-c5f2100bd856"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:50:07.041871 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:07.041797    2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84a151d1-16e9-466d-b684-c5f2100bd856-webhook-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:50:07.041871 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:07.041825    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jp2t\" (UniqueName: \"kubernetes.io/projected/84a151d1-16e9-466d-b684-c5f2100bd856-kube-api-access-4jp2t\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 11:50:07.243337 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:07.243301    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:50:07.246692 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:07.246667    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6954b9867f-m6sh8"]
Jul 28 11:50:08.065165 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.065131    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6954b9867f-tzmg8"]
Jul 28 11:50:08.065537 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.065461    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a151d1-16e9-466d-b684-c5f2100bd856" containerName="manager"
Jul 28 11:50:08.065537 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.065472    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a151d1-16e9-466d-b684-c5f2100bd856" containerName="manager"
Jul 28 11:50:08.065607 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.065548    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a151d1-16e9-466d-b684-c5f2100bd856" containerName="manager"
Jul 28 11:50:08.069782 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.069764    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.071768 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.071743    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 11:50:08.072240 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.072223    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 11:50:08.072325 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.072224    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nprmb\""
Jul 28 11:50:08.077441 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.077415    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6954b9867f-tzmg8"]
Jul 28 11:50:08.108656 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.108625    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a151d1-16e9-466d-b684-c5f2100bd856" path="/var/lib/kubelet/pods/84a151d1-16e9-466d-b684-c5f2100bd856/volumes"
Jul 28 11:50:08.149229 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.149198    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-webhook-cert\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.149350 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.149329    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxw4\" (UniqueName: \"kubernetes.io/projected/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-kube-api-access-4sxw4\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.250302 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.250271    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-webhook-cert\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.250302 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.250304    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxw4\" (UniqueName: \"kubernetes.io/projected/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-kube-api-access-4sxw4\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.252715 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.252690    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-webhook-cert\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.258148 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.258125    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxw4\" (UniqueName: \"kubernetes.io/projected/c24e3e82-e29a-4a5a-b3f4-d09cb354139f-kube-api-access-4sxw4\") pod \"maas-controller-6954b9867f-tzmg8\" (UID: \"c24e3e82-e29a-4a5a-b3f4-d09cb354139f\") " pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.381309 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.381218    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:08.512386 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.512350    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6954b9867f-tzmg8"]
Jul 28 11:50:08.515512 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:50:08.515478    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24e3e82_e29a_4a5a_b3f4_d09cb354139f.slice/crio-4a5d93cb9380e151b30c6f1f712a22de8fc3b6103e0a0b5c181445a466b0b052 WatchSource:0}: Error finding container 4a5d93cb9380e151b30c6f1f712a22de8fc3b6103e0a0b5c181445a466b0b052: Status 404 returned error can't find the container with id 4a5d93cb9380e151b30c6f1f712a22de8fc3b6103e0a0b5c181445a466b0b052
Jul 28 11:50:08.938056 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:08.938018    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-tzmg8" event={"ID":"c24e3e82-e29a-4a5a-b3f4-d09cb354139f","Type":"ContainerStarted","Data":"4a5d93cb9380e151b30c6f1f712a22de8fc3b6103e0a0b5c181445a466b0b052"}
Jul 28 11:50:09.943867 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:09.943830    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6954b9867f-tzmg8" event={"ID":"c24e3e82-e29a-4a5a-b3f4-d09cb354139f","Type":"ContainerStarted","Data":"d18367ccaecc2ff63815d84a594d4c8f12df84222cebfcf6fd591d486eaa7ef3"}
Jul 28 11:50:09.944285 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:09.943944    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:50:09.960325 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:09.960281    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6954b9867f-tzmg8" podStartSLOduration=1.5168705070000001 podStartE2EDuration="1.960254896s" totalImagesPullingTime="443.384389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:50:09.95798405 +0000 UTC m=+1174.415018544" watchObservedRunningTime="2026-07-28 11:50:09.960254896 +0000 UTC m=+1174.417289352"
Jul 28 11:50:20.950239 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:50:20.950194    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6954b9867f-tzmg8"
Jul 28 11:59:17.111619 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.111576    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 11:59:17.114876 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.114855    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.117187 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.117163    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 11:59:17.117303 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.117201    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-637c70a6-kserve-self-signed-certs\""
Jul 28 11:59:17.117667 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.117648    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n92jt\""
Jul 28 11:59:17.117770 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.117690    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 11:59:17.126417 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.126394    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 11:59:17.268198 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268159    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.268392 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268217    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.268392 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268297    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kgb5\" (UniqueName: \"kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.268392 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268371    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.268551 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268396    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.268551 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.268418    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369472 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369377    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369472 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369415    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369472 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369431    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369472 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369479    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369808 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369519    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369808 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369541    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kgb5\" (UniqueName: \"kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.369971 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369941    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.370093 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369980    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.370093 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.369999    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.371688 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.371661    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.372116 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.372093    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.379838 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.379809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kgb5\" (UniqueName: \"kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5\") pod \"e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") " pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.424564 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.424535    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:17.559422 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.559400    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 11:59:17.562083 ip-10-0-130-140 kubenswrapper[2552]: W0728 11:59:17.562056    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod721dbb0d_7837_4cbc_b4e2_dd7fc19f84a3.slice/crio-34668861a4f0459e08b40fda940eda5b50593423e8335b5daa54eda61b983886 WatchSource:0}: Error finding container 34668861a4f0459e08b40fda940eda5b50593423e8335b5daa54eda61b983886: Status 404 returned error can't find the container with id 34668861a4f0459e08b40fda940eda5b50593423e8335b5daa54eda61b983886
Jul 28 11:59:17.564082 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:17.564066    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:59:18.422099 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:18.422063    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" event={"ID":"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3","Type":"ContainerStarted","Data":"34668861a4f0459e08b40fda940eda5b50593423e8335b5daa54eda61b983886"}
Jul 28 11:59:19.428373 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:19.428334    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" event={"ID":"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3","Type":"ContainerStarted","Data":"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"}
Jul 28 11:59:19.447084 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:19.447028    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" podStartSLOduration=1.3482530160000001 podStartE2EDuration="2.447013032s" totalImagesPullingTime="1.098760016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:59:19.445715022 +0000 UTC m=+1723.902749477" watchObservedRunningTime="2026-07-28 11:59:19.447013032 +0000 UTC m=+1723.904047488"
Jul 28 11:59:27.425553 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:27.425514    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:27.426004 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:27.425569    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:27.437806 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:27.437783    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:27.473965 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:27.473939    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 11:59:50.127746 ip-10-0-130-140 kubenswrapper[2552]: I0728 11:59:50.127705    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 12:00:05.147312 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.147258    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" podUID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" containerName="main" containerID="cri-o://244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af" gracePeriod=45
Jul 28 12:00:05.408813 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.408754    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 12:00:05.502789 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.502753    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.502993 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.502819    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kgb5\" (UniqueName: \"kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.502993 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.502875    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.503126 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503094    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.503187 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503164    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.503239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503092    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home" (OuterVolumeSpecName: "home") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:00:05.503239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503200    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir\") pod \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\" (UID: \"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3\") "
Jul 28 12:00:05.503239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503183    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:00:05.503455 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503410    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-home\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.503455 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503431    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-model-cache\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.503647 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.503461    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:00:05.505194 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.505168    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5" (OuterVolumeSpecName: "kube-api-access-8kgb5") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "kube-api-access-8kgb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:00:05.505194 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.505174    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm" (OuterVolumeSpecName: "dshm") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:00:05.505340 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.505235    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" (UID: "721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:00:05.604514 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.604482    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tls-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.604514 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.604509    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-dshm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.604713 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.604538    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-tmp-dir\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.604713 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.604548    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kgb5\" (UniqueName: \"kubernetes.io/projected/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3-kube-api-access-8kgb5\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:00:05.639215 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.639180    2552 generic.go:356] "Generic (PLEG): container finished" podID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" containerID="244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af" exitCode=0
Jul 28 12:00:05.639383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.639223    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" event={"ID":"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3","Type":"ContainerDied","Data":"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"}
Jul 28 12:00:05.639383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.639240    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"
Jul 28 12:00:05.639383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.639260    2552 scope.go:117] "RemoveContainer" containerID="244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"
Jul 28 12:00:05.639383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.639251    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm" event={"ID":"721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3","Type":"ContainerDied","Data":"34668861a4f0459e08b40fda940eda5b50593423e8335b5daa54eda61b983886"}
Jul 28 12:00:05.650277 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.650260    2552 scope.go:117] "RemoveContainer" containerID="244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"
Jul 28 12:00:05.650564 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:00:05.650540    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af\": container with ID starting with 244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af not found: ID does not exist" containerID="244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"
Jul 28 12:00:05.650648 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.650572    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af"} err="failed to get container status \"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af\": rpc error: code = NotFound desc = could not find container \"244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af\": container with ID starting with 244800a8ad7cb4a43db1ca9d009acb89260c8f100e1e3d92447e5898dd5747af not found: ID does not exist"
Jul 28 12:00:05.663688 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.663632    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 12:00:05.667214 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:05.667193    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-637c70a6-kserve-55b4c4fbd7-pk5lm"]
Jul 28 12:00:06.108785 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:00:06.108744    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" path="/var/lib/kubelet/pods/721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3/volumes"
Jul 28 12:03:38.896516 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.896437    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:03:38.899129 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.896826    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" containerName="main"
Jul 28 12:03:38.899129 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.896844    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" containerName="main"
Jul 28 12:03:38.899129 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.896972    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="721dbb0d-7837-4cbc-b4e2-dd7fc19f84a3" containerName="main"
Jul 28 12:03:38.900080 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.900064    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.902193 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.902175    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7e937708-openshift-default-dockercfg-bqk7q\""
Jul 28 12:03:38.927655 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.927626    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:03:38.950092 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.950049    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.950317 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.950296    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.950465 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.950451    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.950572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.950559    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.950964 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.950942    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.951105 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.951089    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.951406 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.951248    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.951496 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.951469    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfszm\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:38.951546 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:38.951516    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052106 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052066    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052117    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfszm\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052145    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052168    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052192    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052229    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052274 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052255    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052288    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052318    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052678 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052564    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052735 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052691    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.052920 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052864    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.053023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.052996    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.053142 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.053122    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.054809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.054782    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.054930 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.054913    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.060731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.060700    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.060998 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.060973    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfszm\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm\") pod \"e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") " pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.209916 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.209860    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:39.355125 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.355092    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:03:39.358417 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:03:39.358382    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28a8ec1_3dc9_44d1_b0d5_824af27dce36.slice/crio-0bafb5552fd0b713b41e8c782ea72373894d0eb756779c7daf51bdb47f786a7b WatchSource:0}: Error finding container 0bafb5552fd0b713b41e8c782ea72373894d0eb756779c7daf51bdb47f786a7b: Status 404 returned error can't find the container with id 0bafb5552fd0b713b41e8c782ea72373894d0eb756779c7daf51bdb47f786a7b
Jul 28 12:03:39.616700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:39.616619    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerStarted","Data":"0bafb5552fd0b713b41e8c782ea72373894d0eb756779c7daf51bdb47f786a7b"}
Jul 28 12:03:42.041669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:42.041630    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:42.041973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:42.041707    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:42.041973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:42.041738    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:42.633330 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:42.633287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerStarted","Data":"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"}
Jul 28 12:03:42.657240 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:42.657191    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podStartSLOduration=1.976643454 podStartE2EDuration="4.657176573s" totalImagesPullingTime="2.680533119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:03:42.655665276 +0000 UTC m=+1987.112699735" watchObservedRunningTime="2026-07-28 12:03:42.657176573 +0000 UTC m=+1987.114211030"
Jul 28 12:03:43.210525 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:43.210488    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:43.215213 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:43.215188    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:43.638354 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:43.638273    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:43.639168 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:43.639149    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:56.699247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.699211    2552 generic.go:356] "Generic (PLEG): container finished" podID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerID="5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac" exitCode=0
Jul 28 12:03:56.699763 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.699287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerDied","Data":"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"}
Jul 28 12:03:56.699834 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.699775    2552 scope.go:117] "RemoveContainer" containerID="5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"
Jul 28 12:03:56.701776 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.701738    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:56.701910 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.701839    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:56.701986 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:56.701909    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:03:57.705557 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:57.705524    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerStarted","Data":"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"}
Jul 28 12:03:58.210807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:58.210778    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:58.211953 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:58.211925    2552 patch_prober.go:28] interesting pod/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 28 12:03:58.212047 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:58.211977    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 28 12:03:58.638756 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:58.638629    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:03:59.211093 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:59.211056    2552 patch_prober.go:28] interesting pod/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 28 12:03:59.211497 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:03:59.211110    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 28 12:04:00.211195 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:00.211152    2552 patch_prober.go:28] interesting pod/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 28 12:04:00.211603 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:00.211218    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 28 12:04:01.210172 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:01.210126    2552 patch_prober.go:28] interesting pod/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 28 12:04:01.210395 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:01.210192    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 28 12:04:02.214982 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:02.214950    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:04:02.216057 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:02.216035    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:04:25.032053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:25.032017    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:04:25.032684 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:25.032316    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy" containerID="cri-o://12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403" gracePeriod=30
Jul 28 12:04:26.327247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.327215    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:04:26.330787 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.330765    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.333336 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:04:26.333308    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-migrate-c98bc341-gw-openshift-default-dockercfg-mxrcp\" is forbidden: User \"system:node:ip-10-0-130-140.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-140.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c98bc341-gw-openshift-default-dockercfg-mxrcp\"" type="*v1.Secret"
Jul 28 12:04:26.333336 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:04:26.333304    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" is forbidden: User \"system:node:ip-10-0-130-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-140.ec2.internal' and this object" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.386283 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.386246    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:04:26.388791 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388763    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.388980 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388806    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v9tj\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.388980 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388833    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.388980 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388858    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.388980 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388924    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.389209 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.388996    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.389209 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.389069    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.389209 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.389093    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.389209 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.389130    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490040 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490006    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490040 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490043    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490067    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490095    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490132    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v9tj\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490167    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490192    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490285    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490441    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490535    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490558    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490833 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.490944 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.490925    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.492660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.492631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.492828 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.492808    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.497318 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.497290    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:26.497502 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:26.497483    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v9tj\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj\") pod \"e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") " pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:27.169957 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.169925    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c98bc341-gw-openshift-default-dockercfg-mxrcp\""
Jul 28 12:04:27.170958 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.170931    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:27.326732 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.326708    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:04:27.328722 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:04:27.328698    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cdb4a0_9d56_4491_bf9d_96989e0775b7.slice/crio-ca17b3059cccda79c692feff54f07517ba42b54edfb9a4d2aef891b88b10872a WatchSource:0}: Error finding container ca17b3059cccda79c692feff54f07517ba42b54edfb9a4d2aef891b88b10872a: Status 404 returned error can't find the container with id ca17b3059cccda79c692feff54f07517ba42b54edfb9a4d2aef891b88b10872a
Jul 28 12:04:27.331369 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.331334    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:27.331459 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.331405    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:27.331459 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.331436    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:27.857001 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.856955    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerStarted","Data":"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"}
Jul 28 12:04:27.857193 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.857024    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerStarted","Data":"ca17b3059cccda79c692feff54f07517ba42b54edfb9a4d2aef891b88b10872a"}
Jul 28 12:04:27.880752 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:27.880697    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podStartSLOduration=1.88068307 podStartE2EDuration="1.88068307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:04:27.879393643 +0000 UTC m=+2032.336428100" watchObservedRunningTime="2026-07-28 12:04:27.88068307 +0000 UTC m=+2032.337717526"
Jul 28 12:04:28.171238 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:28.171147    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:28.175873 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:28.175849    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:28.862038 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:28.861999    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:28.862871 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:28.862854    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:30.290985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.290924    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:04:30.330094 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330064    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330108    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330135    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfszm\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330198    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330228    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330281    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330308    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330338    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330362    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:04:30.330422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330390    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert\") pod \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\" (UID: \"a28a8ec1-3dc9-44d1-b0d5-824af27dce36\") "
Jul 28 12:04:30.330683 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330626    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:04:30.330744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330724    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.330802 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330733    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:04:30.330802 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.330749    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.331112 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.331086    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:04:30.331639 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.331617    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data" (OuterVolumeSpecName: "istio-data") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:04:30.333010 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.332981    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:04:30.333168 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.333142    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:04:30.333262 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.333218    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm" (OuterVolumeSpecName: "kube-api-access-vfszm") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "kube-api-access-vfszm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:04:30.333262 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.333234    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token" (OuterVolumeSpecName: "istio-token") pod "a28a8ec1-3dc9-44d1-b0d5-824af27dce36" (UID: "a28a8ec1-3dc9-44d1-b0d5-824af27dce36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:04:30.432078 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432043    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432078 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432074    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432078 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432084    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432092    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432101    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432109    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.432302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.432118    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfszm\" (UniqueName: \"kubernetes.io/projected/a28a8ec1-3dc9-44d1-b0d5-824af27dce36-kube-api-access-vfszm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:04:30.879351 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.879313    2552 generic.go:356] "Generic (PLEG): container finished" podID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerID="12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403" exitCode=0
Jul 28 12:04:30.879554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.879369    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"
Jul 28 12:04:30.879554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.879404    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerDied","Data":"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"}
Jul 28 12:04:30.879554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.879496    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt" event={"ID":"a28a8ec1-3dc9-44d1-b0d5-824af27dce36","Type":"ContainerDied","Data":"0bafb5552fd0b713b41e8c782ea72373894d0eb756779c7daf51bdb47f786a7b"}
Jul 28 12:04:30.879554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.879529    2552 scope.go:117] "RemoveContainer" containerID="12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"
Jul 28 12:04:30.889169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.889151    2552 scope.go:117] "RemoveContainer" containerID="5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"
Jul 28 12:04:30.897307 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.897286    2552 scope.go:117] "RemoveContainer" containerID="12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"
Jul 28 12:04:30.897585 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:04:30.897566    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403\": container with ID starting with 12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403 not found: ID does not exist" containerID="12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"
Jul 28 12:04:30.897663 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.897597    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403"} err="failed to get container status \"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403\": rpc error: code = NotFound desc = could not find container \"12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403\": container with ID starting with 12d05f0c2530ffc815b87a02e3badb14125f77fdf6c7646cfc02bb8c365ab403 not found: ID does not exist"
Jul 28 12:04:30.897663 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.897621    2552 scope.go:117] "RemoveContainer" containerID="5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"
Jul 28 12:04:30.897878 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:04:30.897860    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac\": container with ID starting with 5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac not found: ID does not exist" containerID="5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"
Jul 28 12:04:30.897932 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.897884    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac"} err="failed to get container status \"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac\": rpc error: code = NotFound desc = could not find container \"5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac\": container with ID starting with 5416989819cba034960e2adf97913c4648960de4224de8a4b2f4ecb5b5a8ecac not found: ID does not exist"
Jul 28 12:04:30.904387 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.904365    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:04:30.908625 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:30.908603    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7e937708-openshift-default-786cc864dd-jjxnt"]
Jul 28 12:04:32.109209 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:32.109178    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" path="/var/lib/kubelet/pods/a28a8ec1-3dc9-44d1-b0d5-824af27dce36/volumes"
Jul 28 12:04:40.928345 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.928309    2552 generic.go:356] "Generic (PLEG): container finished" podID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerID="d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c" exitCode=0
Jul 28 12:04:40.928760 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.928378    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerDied","Data":"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"}
Jul 28 12:04:40.928760 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.928724    2552 scope.go:117] "RemoveContainer" containerID="d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"
Jul 28 12:04:40.929766 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.929735    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:40.929841 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.929814    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:40.929877 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:40.929850    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:04:41.939076 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:41.939022    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerStarted","Data":"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"}
Jul 28 12:04:42.172000 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:42.171954    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:42.173191 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:42.173162    2552 patch_prober.go:28] interesting pod/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 28 12:04:42.173317 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:42.173234    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 28 12:04:43.171779 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:43.171743    2552 patch_prober.go:28] interesting pod/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 28 12:04:43.172208 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:43.171817    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 28 12:04:43.862083 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:43.862027    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:44.171631 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:44.171541    2552 patch_prober.go:28] interesting pod/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 28 12:04:44.171631 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:44.171601    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 28 12:04:45.171344 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:45.171303    2552 patch_prober.go:28] interesting pod/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 28 12:04:45.171733 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:45.171365    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 28 12:04:46.175335 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:46.175303    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:04:46.176409 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:04:46.176388    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:05:09.984178 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:09.984141    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:05:09.984775 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:09.984423    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy" containerID="cri-o://2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c" gracePeriod=30
Jul 28 12:05:10.856642 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.856606    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:05:10.857267 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.857243    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:10.857267 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.857267    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:10.857461 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.857394    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:10.857461 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.857406    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:10.860566 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.860544    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.863061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.863040    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ae7b06d1-openshift-default-dockercfg-w747b\""
Jul 28 12:05:10.874288 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.874262    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:05:10.905595 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905563    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqtm\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905614    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905645    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905673    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905704    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905738    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905763    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905800    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:10.905985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:10.905830    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.006810 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006769    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.006810 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006815    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006863    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006883    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006940    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.006972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007118    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqtm\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007160    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007383 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007376    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007811 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007400    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007811 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007534    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.007811 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.007537    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.008201 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.008172    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.009479 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.009454    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.010533 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.010508    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.014696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.014667    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.014808 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.014700    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqtm\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm\") pod \"e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") " pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.172281 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.172186    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:11.327126 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.327099    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:05:11.329065 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:05:11.329037    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07dc162_f207_48ca_b7d1_66f678661c12.slice/crio-2aa744ea83dbea785ed43db090c3eb3dafac67d349f36d309705337e71046532 WatchSource:0}: Error finding container 2aa744ea83dbea785ed43db090c3eb3dafac67d349f36d309705337e71046532: Status 404 returned error can't find the container with id 2aa744ea83dbea785ed43db090c3eb3dafac67d349f36d309705337e71046532
Jul 28 12:05:11.331178 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.331146    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:11.331286 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.331208    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:11.331286 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:11.331235    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:12.082099 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:12.082060    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerStarted","Data":"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"}
Jul 28 12:05:12.082524 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:12.082116    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerStarted","Data":"2aa744ea83dbea785ed43db090c3eb3dafac67d349f36d309705337e71046532"}
Jul 28 12:05:12.102945 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:12.102875    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podStartSLOduration=2.102857843 podStartE2EDuration="2.102857843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:05:12.102032279 +0000 UTC m=+2076.559066737" watchObservedRunningTime="2026-07-28 12:05:12.102857843 +0000 UTC m=+2076.559892300"
Jul 28 12:05:12.172873 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:12.172842    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:12.178267 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:12.178241    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:13.086607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:13.086572    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:13.087489 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:13.087466    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:15.233681 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.233658    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:05:15.346318 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346234    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346318 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346267    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346306    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346363    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346387    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346410    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346475    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v9tj\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346502    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346856 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346561    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token\") pod \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\" (UID: \"24cdb4a0-9d56-4491-bf9d-96989e0775b7\") "
Jul 28 12:05:15.346856 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346725    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:05:15.346856 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346744    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:05:15.347154 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346745    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:05:15.347154 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346936    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.347154 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346950    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:05:15.347154 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.346959    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.347370 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.347150    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:05:15.348833 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.348807    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:05:15.349014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.348812    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:05:15.349149 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.349130    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:05:15.349206 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.349144    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj" (OuterVolumeSpecName: "kube-api-access-4v9tj") pod "24cdb4a0-9d56-4491-bf9d-96989e0775b7" (UID: "24cdb4a0-9d56-4491-bf9d-96989e0775b7"). InnerVolumeSpecName "kube-api-access-4v9tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:05:15.448377 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448345    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v9tj\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-kube-api-access-4v9tj\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448377 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448375    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448377 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448385    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448598 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448393    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448598 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448401    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448598 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448409    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:15.448598 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:15.448417    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24cdb4a0-9d56-4491-bf9d-96989e0775b7-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:05:16.101018 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.100979    2552 generic.go:356] "Generic (PLEG): container finished" podID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerID="2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c" exitCode=0
Jul 28 12:05:16.101169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.101033    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"
Jul 28 12:05:16.101169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.101048    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerDied","Data":"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"}
Jul 28 12:05:16.101169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.101101    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh" event={"ID":"24cdb4a0-9d56-4491-bf9d-96989e0775b7","Type":"ContainerDied","Data":"ca17b3059cccda79c692feff54f07517ba42b54edfb9a4d2aef891b88b10872a"}
Jul 28 12:05:16.101169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.101131    2552 scope.go:117] "RemoveContainer" containerID="2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"
Jul 28 12:05:16.112063 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.111814    2552 scope.go:117] "RemoveContainer" containerID="d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"
Jul 28 12:05:16.120045 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.120024    2552 scope.go:117] "RemoveContainer" containerID="2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"
Jul 28 12:05:16.120292 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:05:16.120271    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c\": container with ID starting with 2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c not found: ID does not exist" containerID="2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"
Jul 28 12:05:16.120361 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.120299    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c"} err="failed to get container status \"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c\": rpc error: code = NotFound desc = could not find container \"2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c\": container with ID starting with 2bf774d258914f3283aecc3adbbdd76c6da6f4ccb367e6ebc9864916b585fd6c not found: ID does not exist"
Jul 28 12:05:16.120361 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.120316    2552 scope.go:117] "RemoveContainer" containerID="d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"
Jul 28 12:05:16.120542 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:05:16.120524    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c\": container with ID starting with d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c not found: ID does not exist" containerID="d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"
Jul 28 12:05:16.120579 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.120546    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c"} err="failed to get container status \"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c\": rpc error: code = NotFound desc = could not find container \"d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c\": container with ID starting with d10a9e0fa6f3812577e5afed52a332ad6bfee512cddeddc0365c73809e96129c not found: ID does not exist"
Jul 28 12:05:16.128571 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.128541    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:05:16.130562 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:16.130543    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c98bc341-gw-openshift-default-d99779cfb-md8rh"]
Jul 28 12:05:18.108410 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:18.108371    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" path="/var/lib/kubelet/pods/24cdb4a0-9d56-4491-bf9d-96989e0775b7/volumes"
Jul 28 12:05:28.087874 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.087834    2552 patch_prober.go:28] interesting pod/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 12:05:28.088303 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.087919    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 12:05:28.157558 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.157522    2552 generic.go:356] "Generic (PLEG): container finished" podID="b07dc162-f207-48ca-b7d1-66f678661c12" containerID="73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a" exitCode=0
Jul 28 12:05:28.157750 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.157562    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerDied","Data":"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"}
Jul 28 12:05:28.158031 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.158012    2552 scope.go:117] "RemoveContainer" containerID="73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"
Jul 28 12:05:28.159103 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.159070    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:28.159197 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.159146    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:28.159197 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:28.159179    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:29.165072 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:29.164992    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerStarted","Data":"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"}
Jul 28 12:05:29.173108 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:29.173085    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:29.173271 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:29.173246    2552 patch_prober.go:28] interesting pod/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 12:05:29.173340 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:29.173303    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 12:05:30.173113 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:30.173080    2552 patch_prober.go:28] interesting pod/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 12:05:30.173536 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:30.173141    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 12:05:31.172545 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:31.172504    2552 patch_prober.go:28] interesting pod/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 12:05:31.172803 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:31.172564    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 12:05:32.173508 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:32.173468    2552 patch_prober.go:28] interesting pod/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 12:05:32.173939 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:32.173531    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 12:05:33.177066 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:33.177027    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:33.177534 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:33.177303    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:33.178105 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:33.178086    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:05:55.763687 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:55.763650    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:05:55.764213 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:55.763888    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy" containerID="cri-o://58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9" gracePeriod=30
Jul 28 12:05:56.648715 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.648681    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:05:56.649087 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649074    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:56.649140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649089    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28a8ec1-3dc9-44d1-b0d5-824af27dce36" containerName="istio-proxy"
Jul 28 12:05:56.649140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649123    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:05:56.649140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649129    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:05:56.649140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649136    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:05:56.649140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649141    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:05:56.649342 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.649201    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:05:56.652263 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.652243    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.654343 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.654326    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9636f5f9-openshift-default-dockercfg-ttb8h\""
Jul 28 12:05:56.666732 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.666708    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:05:56.721466 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721432    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721479    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721514    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721537    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721574    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721601    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721942 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721690    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721942 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721726    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9bms\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.721942 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.721790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.822539 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822498    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.822539 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822540    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822606    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822639    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9bms\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822682    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822730    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823053 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.822760    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823139    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823128    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823047    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823188    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823247 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823220    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823439 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823423    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.823955 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.823933    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.824961 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.824939    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.825028 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.824962    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.837175 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.837149    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.837436 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.837414    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9bms\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms\") pod \"e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") " pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:56.965985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:56.965950    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:57.105556 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.105531    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:05:57.111284 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.111245    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:57.111412 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.111312    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:57.111412 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.111339    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:05:57.298834 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.298741    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerStarted","Data":"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"}
Jul 28 12:05:57.298834 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.298803    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerStarted","Data":"dc96e6fd407c3a76574ccc7648f1e60d8e269079d0a47d900c9a43ab8ac2253d"}
Jul 28 12:05:57.326453 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.326394    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podStartSLOduration=1.326372952 podStartE2EDuration="1.326372952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:05:57.325917782 +0000 UTC m=+2121.782952247" watchObservedRunningTime="2026-07-28 12:05:57.326372952 +0000 UTC m=+2121.783407410"
Jul 28 12:05:57.966473 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:57.966444    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:58.971486 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:58.971451    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:59.309259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:59.309170    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:05:59.310285 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:05:59.310264    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:01.025484 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.025460    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:06:01.062818 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062746    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.062818 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062792    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062853    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062876    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062945    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.062967    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063034    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063283 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063073    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqqtm\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063283 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063097    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs\") pod \"b07dc162-f207-48ca-b7d1-66f678661c12\" (UID: \"b07dc162-f207-48ca-b7d1-66f678661c12\") "
Jul 28 12:06:01.063283 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063124    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:01.063437 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063374    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.063493 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063435    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:06:01.063549 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063497    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:01.063644 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063618    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:01.063833 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.063769    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data" (OuterVolumeSpecName: "istio-data") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:01.065182 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.065160    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:06:01.065339 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.065322    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token" (OuterVolumeSpecName: "istio-token") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:06:01.065494 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.065470    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm" (OuterVolumeSpecName: "kube-api-access-zqqtm") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "kube-api-access-zqqtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:06:01.065675 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.065655    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b07dc162-f207-48ca-b7d1-66f678661c12" (UID: "b07dc162-f207-48ca-b7d1-66f678661c12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:01.163776 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163741    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07dc162-f207-48ca-b7d1-66f678661c12-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.163776 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163770    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.163776 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163779    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.164014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163790    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.164014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163799    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07dc162-f207-48ca-b7d1-66f678661c12-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.164014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163807    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqqtm\" (UniqueName: \"kubernetes.io/projected/b07dc162-f207-48ca-b7d1-66f678661c12-kube-api-access-zqqtm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.164014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163815    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.164014 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.163823    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07dc162-f207-48ca-b7d1-66f678661c12-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:01.319459 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.319369    2552 generic.go:356] "Generic (PLEG): container finished" podID="b07dc162-f207-48ca-b7d1-66f678661c12" containerID="58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9" exitCode=0
Jul 28 12:06:01.319459 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.319426    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"
Jul 28 12:06:01.319700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.319426    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerDied","Data":"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"}
Jul 28 12:06:01.319700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.319559    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs" event={"ID":"b07dc162-f207-48ca-b7d1-66f678661c12","Type":"ContainerDied","Data":"2aa744ea83dbea785ed43db090c3eb3dafac67d349f36d309705337e71046532"}
Jul 28 12:06:01.319700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.319586    2552 scope.go:117] "RemoveContainer" containerID="58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"
Jul 28 12:06:01.328686 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.328670    2552 scope.go:117] "RemoveContainer" containerID="73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"
Jul 28 12:06:01.336319 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.336301    2552 scope.go:117] "RemoveContainer" containerID="58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"
Jul 28 12:06:01.336544 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:06:01.336525    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9\": container with ID starting with 58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9 not found: ID does not exist" containerID="58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"
Jul 28 12:06:01.336591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.336550    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9"} err="failed to get container status \"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9\": rpc error: code = NotFound desc = could not find container \"58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9\": container with ID starting with 58e5137593dfd57b9d1a342335696125cfe638db24f9def842d7b1a68b9d69d9 not found: ID does not exist"
Jul 28 12:06:01.336591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.336564    2552 scope.go:117] "RemoveContainer" containerID="73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"
Jul 28 12:06:01.336744 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:06:01.336729    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a\": container with ID starting with 73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a not found: ID does not exist" containerID="73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"
Jul 28 12:06:01.336789 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.336748    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a"} err="failed to get container status \"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a\": rpc error: code = NotFound desc = could not find container \"73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a\": container with ID starting with 73c5b3d7ed29874e89a84801dfa7aeae8909fd32709e311333d896f931893b6a not found: ID does not exist"
Jul 28 12:06:01.343805 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.343773    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:06:01.348060 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:01.348039    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ae7b06d1-openshift-default-566774cfd8-g4qqs"]
Jul 28 12:06:02.109196 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:02.109160    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" path="/var/lib/kubelet/pods/b07dc162-f207-48ca-b7d1-66f678661c12/volumes"
Jul 28 12:06:10.366948 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.366908    2552 generic.go:356] "Generic (PLEG): container finished" podID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerID="f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba" exitCode=0
Jul 28 12:06:10.367386 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.366966    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerDied","Data":"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"}
Jul 28 12:06:10.367386 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.367326    2552 scope.go:117] "RemoveContainer" containerID="f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"
Jul 28 12:06:10.368413 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.368379    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:06:10.368480 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.368458    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:06:10.368522 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:10.368490    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:06:11.373109 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:11.373075    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerStarted","Data":"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"}
Jul 28 12:06:11.966622 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:11.966584    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:11.967687 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:11.967660    2552 patch_prober.go:28] interesting pod/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 28 12:06:11.967754 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:11.967715    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 28 12:06:12.966885 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:12.966846    2552 patch_prober.go:28] interesting pod/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 28 12:06:12.967324 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:12.966946    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 28 12:06:13.966392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:13.966350    2552 patch_prober.go:28] interesting pod/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 28 12:06:13.966583 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:13.966427    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 28 12:06:14.310071 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:14.309973    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:14.966359 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:14.966318    2552 patch_prober.go:28] interesting pod/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 28 12:06:14.966564 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:14.966392    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 28 12:06:15.971051 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:15.971010    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:15.971914 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:15.971883    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:38.797284 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:38.797244    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:06:38.797752 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:38.797497    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy" containerID="cri-o://3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda" gracePeriod=30
Jul 28 12:06:44.061243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.061219    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154254    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154328    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154386    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154439    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154479    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9bms\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154521    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154556    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154614    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.154966 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.154648    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket\") pod \"7e4552e6-a954-461c-b15c-0e98d3c6a801\" (UID: \"7e4552e6-a954-461c-b15c-0e98d3c6a801\") "
Jul 28 12:06:44.161404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.156773    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:06:44.161404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.157138    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:06:44.161404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.157365    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:44.161404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.157553    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:44.161404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.161006    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:06:44.162017 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.161992    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:44.162017 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.161995    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms" (OuterVolumeSpecName: "kube-api-access-l9bms") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "kube-api-access-l9bms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:06:44.162182 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.162136    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:44.163115 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.163088    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e4552e6-a954-461c-b15c-0e98d3c6a801" (UID: "7e4552e6-a954-461c-b15c-0e98d3c6a801"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:06:44.255864 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255832    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.255864 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255858    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9bms\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-kube-api-access-l9bms\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.255864 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255870    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255879    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4552e6-a954-461c-b15c-0e98d3c6a801-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255916    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255926    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255934    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255942    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4552e6-a954-461c-b15c-0e98d3c6a801-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.256111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.255950    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4552e6-a954-461c-b15c-0e98d3c6a801-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:06:44.525563 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.525531    2552 generic.go:356] "Generic (PLEG): container finished" podID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerID="3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda" exitCode=0
Jul 28 12:06:44.525740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.525597    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"
Jul 28 12:06:44.525740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.525599    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerDied","Data":"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"}
Jul 28 12:06:44.525847 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.525740    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c" event={"ID":"7e4552e6-a954-461c-b15c-0e98d3c6a801","Type":"ContainerDied","Data":"dc96e6fd407c3a76574ccc7648f1e60d8e269079d0a47d900c9a43ab8ac2253d"}
Jul 28 12:06:44.525847 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.525762    2552 scope.go:117] "RemoveContainer" containerID="3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"
Jul 28 12:06:44.534510 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.534492    2552 scope.go:117] "RemoveContainer" containerID="f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"
Jul 28 12:06:44.542246 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.542230    2552 scope.go:117] "RemoveContainer" containerID="3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"
Jul 28 12:06:44.542496 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:06:44.542476    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda\": container with ID starting with 3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda not found: ID does not exist" containerID="3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"
Jul 28 12:06:44.542535 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.542505    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda"} err="failed to get container status \"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda\": rpc error: code = NotFound desc = could not find container \"3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda\": container with ID starting with 3c622c4185eff2bafd2d2343d8e8bef69b130b0c547f652e0c783057cb46edda not found: ID does not exist"
Jul 28 12:06:44.542535 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.542523    2552 scope.go:117] "RemoveContainer" containerID="f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"
Jul 28 12:06:44.542752 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:06:44.542737    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba\": container with ID starting with f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba not found: ID does not exist" containerID="f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"
Jul 28 12:06:44.542788 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.542758    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba"} err="failed to get container status \"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba\": rpc error: code = NotFound desc = could not find container \"f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba\": container with ID starting with f67a126a3231a218133fab3d943a0b349f8afa498bdcf7be1d4861bb27a766ba not found: ID does not exist"
Jul 28 12:06:44.546089 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.546068    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:06:44.552044 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:44.552023    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9636f5f9-openshift-default-5748f5d956-mj62c"]
Jul 28 12:06:46.108787 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:06:46.108752    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" path="/var/lib/kubelet/pods/7e4552e6-a954-461c-b15c-0e98d3c6a801/volumes"
Jul 28 12:08:33.013827 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.013785    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014280    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014296    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014310    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014316    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014328    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014333    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014385    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014391    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b07dc162-f207-48ca-b7d1-66f678661c12" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014403    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="24cdb4a0-9d56-4491-bf9d-96989e0775b7" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014410    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:08:33.014444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.014415    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:08:33.017540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.017516    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.019759 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.019740    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9055f75e-openshift-default-dockercfg-st2dd\""
Jul 28 12:08:33.041868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.041843    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:08:33.094755 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094718    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xc92\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094755 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094761    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094987 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094786    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094987 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094850    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094987 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094887    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094987 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094946    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.094987 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.094974    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.095168 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.095021    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.095168 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.095051    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196459 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196416    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xc92\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196482    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196518    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196533    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196555    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196585    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.196660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196614    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196681    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.196724    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.197011    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197176 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.197154    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.197217    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.197372    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.197779 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.197761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.199087 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.199065    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.199196 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.199173    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.207604 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.207579    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xc92\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.207963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.207937    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token\") pod \"e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") " pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.328883 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.328803    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:33.468300 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.468153    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:08:33.471082 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:08:33.471055    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c757225_58ed_4d94_b9f4_8451ffb25643.slice/crio-c842087d8881dfe32402c5ef4d267f5f562bdccc2eeed2c18b81225e5c22a54c WatchSource:0}: Error finding container c842087d8881dfe32402c5ef4d267f5f562bdccc2eeed2c18b81225e5c22a54c: Status 404 returned error can't find the container with id c842087d8881dfe32402c5ef4d267f5f562bdccc2eeed2c18b81225e5c22a54c
Jul 28 12:08:33.473136 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.473099    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:33.473203 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.473175    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:33.473240 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:33.473218    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:34.025549 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:34.025513    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerStarted","Data":"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"}
Jul 28 12:08:34.025978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:34.025571    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerStarted","Data":"c842087d8881dfe32402c5ef4d267f5f562bdccc2eeed2c18b81225e5c22a54c"}
Jul 28 12:08:34.047823 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:34.047776    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podStartSLOduration=2.047761072 podStartE2EDuration="2.047761072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:08:34.046290139 +0000 UTC m=+2278.503324596" watchObservedRunningTime="2026-07-28 12:08:34.047761072 +0000 UTC m=+2278.504795529"
Jul 28 12:08:34.329083 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:34.329002    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:34.333562 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:34.333537    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:35.030755 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:35.030719    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:35.031591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:35.031574    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:50.032338 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.032300    2552 patch_prober.go:28] interesting pod/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 28 12:08:50.032757 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.032357    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 28 12:08:50.101977 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.101944    2552 generic.go:356] "Generic (PLEG): container finished" podID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerID="e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339" exitCode=0
Jul 28 12:08:50.102147 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.102018    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerDied","Data":"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"}
Jul 28 12:08:50.102496 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.102479    2552 scope.go:117] "RemoveContainer" containerID="e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"
Jul 28 12:08:50.103480 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.103453    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:50.103536 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.103520    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:50.103573 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:50.103561    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:08:51.108311 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:51.108273    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerStarted","Data":"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"}
Jul 28 12:08:51.329489 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:51.329454    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:51.329666 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:51.329578    2552 patch_prober.go:28] interesting pod/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 28 12:08:51.329666 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:51.329620    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 28 12:08:52.329437 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:52.329400    2552 patch_prober.go:28] interesting pod/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 28 12:08:52.329843 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:52.329469    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 28 12:08:53.329694 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:53.329660    2552 patch_prober.go:28] interesting pod/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 28 12:08:53.330104 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:53.329747    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 28 12:08:54.329175 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:54.329138    2552 patch_prober.go:28] interesting pod/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 28 12:08:54.329335 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:54.329195    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 28 12:08:55.333406 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:55.333376    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:55.333879 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:55.333632    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:08:55.334514 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:08:55.334496    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:09:10.673841 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:10.673782    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:09:10.674390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:10.674101    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy" containerID="cri-o://d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f" gracePeriod=30
Jul 28 12:09:15.930725 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.930697    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:09:15.983709 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983678    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983716    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983734    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983756    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983786    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983823    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.983919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983907    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.984242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.983996    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xc92\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.984242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984025    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert\") pod \"3c757225-58ed-4d94-b9f4-8451ffb25643\" (UID: \"3c757225-58ed-4d94-b9f4-8451ffb25643\") "
Jul 28 12:09:15.984242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984116    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:09:15.984242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984220    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:09:15.984414 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984240    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:09:15.984414 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984379    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:15.984414 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984399    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:15.984568 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984429    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:15.984568 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984467    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:09:15.984739 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.984716    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:09:15.986076 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.986048    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:09:15.986313 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.986282    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:09:15.986313 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.986292    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:09:15.986478 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:15.986330    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92" (OuterVolumeSpecName: "kube-api-access-8xc92") pod "3c757225-58ed-4d94-b9f4-8451ffb25643" (UID: "3c757225-58ed-4d94-b9f4-8451ffb25643"). InnerVolumeSpecName "kube-api-access-8xc92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:09:16.085705 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085668    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xc92\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-kube-api-access-8xc92\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.085705 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085702    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c757225-58ed-4d94-b9f4-8451ffb25643-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.085705 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085712    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.085968 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085720    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.085968 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085730    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.085968 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.085737    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c757225-58ed-4d94-b9f4-8451ffb25643-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:09:16.219114 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.219079    2552 generic.go:356] "Generic (PLEG): container finished" podID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerID="d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f" exitCode=0
Jul 28 12:09:16.219287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.219136    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"
Jul 28 12:09:16.219287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.219166    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerDied","Data":"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"}
Jul 28 12:09:16.219287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.219231    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d" event={"ID":"3c757225-58ed-4d94-b9f4-8451ffb25643","Type":"ContainerDied","Data":"c842087d8881dfe32402c5ef4d267f5f562bdccc2eeed2c18b81225e5c22a54c"}
Jul 28 12:09:16.219287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.219265    2552 scope.go:117] "RemoveContainer" containerID="d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"
Jul 28 12:09:16.228032 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.228010    2552 scope.go:117] "RemoveContainer" containerID="e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"
Jul 28 12:09:16.236719 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.236697    2552 scope.go:117] "RemoveContainer" containerID="d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"
Jul 28 12:09:16.237042 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:09:16.237013    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f\": container with ID starting with d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f not found: ID does not exist" containerID="d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"
Jul 28 12:09:16.237116 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.237044    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f"} err="failed to get container status \"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f\": rpc error: code = NotFound desc = could not find container \"d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f\": container with ID starting with d57f47ee32b9bd8b793f9630a676feae07a258e623d5ed4cba847b6affab9a1f not found: ID does not exist"
Jul 28 12:09:16.237116 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.237064    2552 scope.go:117] "RemoveContainer" containerID="e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"
Jul 28 12:09:16.237326 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:09:16.237310    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339\": container with ID starting with e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339 not found: ID does not exist" containerID="e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"
Jul 28 12:09:16.237370 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.237331    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339"} err="failed to get container status \"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339\": rpc error: code = NotFound desc = could not find container \"e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339\": container with ID starting with e3b332f45751005e5b5b8badbdd2a3612eeaca30734a7336c883835aa36c4339 not found: ID does not exist"
Jul 28 12:09:16.237973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.237954    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:09:16.241473 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:16.241449    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9055f75e-openshift-default-6897998b88-wkr9d"]
Jul 28 12:09:18.108557 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:18.108521    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" path="/var/lib/kubelet/pods/3c757225-58ed-4d94-b9f4-8451ffb25643/volumes"
Jul 28 12:09:32.170721 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.170460    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:09:32.171729 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.171702    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:09:32.171859 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.171848    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4552e6-a954-461c-b15c-0e98d3c6a801" containerName="istio-proxy"
Jul 28 12:09:32.172028 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.172015    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:09:32.172105 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.172096    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:09:32.172188 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.172179    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:09:32.172254 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.172246    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:09:32.172423 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.172413    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:09:32.177192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.177171    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.179376 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.179356    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7038fa7e-openshift-default-dockercfg-67z6s\""
Jul 28 12:09:32.186178 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.186153    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:09:32.323199 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323163    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323207    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vfm\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323250    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323289    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323352    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323571 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323440    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323571 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323473    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323650 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323566    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.323650 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.323604    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424071 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.423977    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424071 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424027    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424071 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424053    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424105    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424128    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424160    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424190    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vfm\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424221    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424258    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.424810 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.424667    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.425183 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.425151    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.425183 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.425168    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.425445 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.425420    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.425515 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.425470    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.426658 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.426631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.426808 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.426791    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.436700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.436679    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.436990 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.436968    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vfm\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm\") pod \"e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") " pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.489008 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.488979    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:32.631211 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.631185    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:09:32.634842 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:09:32.634813    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a5481c_1634_4c12_abb3_f81ea8e3ef67.slice/crio-083b6a755258c96f61ad2061c7f04883a393d8274f82489d4c6245bc357b31f8 WatchSource:0}: Error finding container 083b6a755258c96f61ad2061c7f04883a393d8274f82489d4c6245bc357b31f8: Status 404 returned error can't find the container with id 083b6a755258c96f61ad2061c7f04883a393d8274f82489d4c6245bc357b31f8
Jul 28 12:09:32.642947 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.642907    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:32.643038 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.642980    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:32.643038 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:32.643008    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:33.304124 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:33.304088    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerStarted","Data":"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"}
Jul 28 12:09:33.304613 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:33.304179    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerStarted","Data":"083b6a755258c96f61ad2061c7f04883a393d8274f82489d4c6245bc357b31f8"}
Jul 28 12:09:33.326589 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:33.326539    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podStartSLOduration=1.326524224 podStartE2EDuration="1.326524224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:09:33.325068314 +0000 UTC m=+2337.782103009" watchObservedRunningTime="2026-07-28 12:09:33.326524224 +0000 UTC m=+2337.783558680"
Jul 28 12:09:33.489835 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:33.489800    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:33.494588 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:33.494564    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:34.308146 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:34.308115    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:34.309068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:34.309050    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:49.309554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.309518    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:49.309954 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.309584    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:49.382496 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.382422    2552 generic.go:356] "Generic (PLEG): container finished" podID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerID="c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d" exitCode=0
Jul 28 12:09:49.382496 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.382482    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerDied","Data":"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"}
Jul 28 12:09:49.382869 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.382854    2552 scope.go:117] "RemoveContainer" containerID="c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"
Jul 28 12:09:49.383906 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.383853    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:49.383984 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.383953    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:49.384031 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:49.383987    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:09:50.389335 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:50.389301    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerStarted","Data":"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"}
Jul 28 12:09:50.489808 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:50.489763    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:50.490078 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:50.490051    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:50.490135 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:50.490118    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:51.489508 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:51.489474    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:51.489935 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:51.489537    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:52.490026 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:52.489988    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:52.490432 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:52.490048    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:53.489758 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:53.489702    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:53.489965 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:53.489782    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:54.489733 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:54.489696    2552 patch_prober.go:28] interesting pod/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 28 12:09:54.490203 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:54.489771    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 28 12:09:55.493144 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:55.493112    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:55.493567 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:55.493393    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:09:55.494110 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:09:55.494093    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:11:32.367826 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:32.367782    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:11:32.368374 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:32.368107    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy" containerID="cri-o://2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc" gracePeriod=30
Jul 28 12:11:37.631696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.631664    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:11:37.720746 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.720715    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.720918 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.720761    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.720918 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.720781    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.720988    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4vfm\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721044    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721102 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721077    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:11:37.721156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721105    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721143    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721255 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721173    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721255 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721166    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:11:37.721255 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721225    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs\") pod \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\" (UID: \"28a5481c-1634-4c12-abb3-f81ea8e3ef67\") "
Jul 28 12:11:37.721591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721480    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:11:37.721591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721517    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.721591 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721543    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.721848 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721651    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:11:37.721930 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.721911    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data" (OuterVolumeSpecName: "istio-data") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:11:37.723170 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.723142    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:11:37.723295 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.723202    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token" (OuterVolumeSpecName: "istio-token") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:11:37.723436 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.723418    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm" (OuterVolumeSpecName: "kube-api-access-g4vfm") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "kube-api-access-g4vfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:11:37.723504 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.723485    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28a5481c-1634-4c12-abb3-f81ea8e3ef67" (UID: "28a5481c-1634-4c12-abb3-f81ea8e3ef67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:11:37.822077 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822044    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822077 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822074    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822077 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822084    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4vfm\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-kube-api-access-g4vfm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822093    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822101    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822109    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.822302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.822116    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a5481c-1634-4c12-abb3-f81ea8e3ef67-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:11:37.879313 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.879234    2552 generic.go:356] "Generic (PLEG): container finished" podID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerID="2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc" exitCode=0
Jul 28 12:11:37.879449 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.879313    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerDied","Data":"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"}
Jul 28 12:11:37.879449 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.879365    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84" event={"ID":"28a5481c-1634-4c12-abb3-f81ea8e3ef67","Type":"ContainerDied","Data":"083b6a755258c96f61ad2061c7f04883a393d8274f82489d4c6245bc357b31f8"}
Jul 28 12:11:37.879449 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.879292    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"
Jul 28 12:11:37.879449 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.879387    2552 scope.go:117] "RemoveContainer" containerID="2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"
Jul 28 12:11:37.888831 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.888813    2552 scope.go:117] "RemoveContainer" containerID="c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"
Jul 28 12:11:37.896417 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.896398    2552 scope.go:117] "RemoveContainer" containerID="2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"
Jul 28 12:11:37.896657 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:11:37.896641    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc\": container with ID starting with 2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc not found: ID does not exist" containerID="2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"
Jul 28 12:11:37.896699 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.896665    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc"} err="failed to get container status \"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc\": rpc error: code = NotFound desc = could not find container \"2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc\": container with ID starting with 2e685e9e63d6116e651b776bf89ef4060fd185c2061a67125c4ef974fadf95fc not found: ID does not exist"
Jul 28 12:11:37.896748 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.896680    2552 scope.go:117] "RemoveContainer" containerID="c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"
Jul 28 12:11:37.896938 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:11:37.896917    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d\": container with ID starting with c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d not found: ID does not exist" containerID="c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"
Jul 28 12:11:37.896992 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.896945    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d"} err="failed to get container status \"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d\": rpc error: code = NotFound desc = could not find container \"c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d\": container with ID starting with c97e1ba217495dbccf374be3cade5b3d1fb10f090a2854fbb88bc1ea6b90930d not found: ID does not exist"
Jul 28 12:11:37.903798 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.903775    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:11:37.913420 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:37.913397    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7038fa7e-openshift-default-6d4dfb6f7b-btv84"]
Jul 28 12:11:38.109860 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:11:38.109768    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" path="/var/lib/kubelet/pods/28a5481c-1634-4c12-abb3-f81ea8e3ef67/volumes"
Jul 28 12:12:29.726330 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.726248    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:12:29.726792 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.726622    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:29.726792 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.726634    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:29.726792 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.726718    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:29.726792 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.726727    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c757225-58ed-4d94-b9f4-8451ffb25643" containerName="istio-proxy"
Jul 28 12:12:29.729695 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.729677    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.731687 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.731668    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-21898b-openshift-default-dockercfg-wvx56\""
Jul 28 12:12:29.740617 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.740587    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:12:29.762298 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762253    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762298 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762302    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762523 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762355    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762523 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762400    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smtx\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762523 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762498    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762543    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762578    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762624    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.762696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.762655    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863413 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863377    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863413 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863415    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863433    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863554    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863586    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863628    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863714    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863775    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smtx\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.863963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.863845    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.864207 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.864072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.864207 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.864180    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.864330 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.864202    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.864561 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.864531    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.866160 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.866130    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.866505 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.866488    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.873706 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.873467    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smtx\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:29.873706 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:29.873665    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token\") pod \"e2e-inf-a-21898b-openshift-default-659677d958-z7lmr\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") " pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:30.041733 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:30.041645    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:30.182584 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:30.182261    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:12:30.185310 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:12:30.185267    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f05b2e_f3a5_4a02_a28c_f0b13e141a8f.slice/crio-d0fdcbc1aff2ec8fc01607c51fc00863e3e2e2b74387bea6e60636cb8d267c26 WatchSource:0}: Error finding container d0fdcbc1aff2ec8fc01607c51fc00863e3e2e2b74387bea6e60636cb8d267c26: Status 404 returned error can't find the container with id d0fdcbc1aff2ec8fc01607c51fc00863e3e2e2b74387bea6e60636cb8d267c26
Jul 28 12:12:30.187986 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:30.187945    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:30.188117 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:30.188018    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:30.188117 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:30.188050    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:31.128831 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:31.128793    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerStarted","Data":"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"}
Jul 28 12:12:31.129263 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:31.128847    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerStarted","Data":"d0fdcbc1aff2ec8fc01607c51fc00863e3e2e2b74387bea6e60636cb8d267c26"}
Jul 28 12:12:31.148622 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:31.148572    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podStartSLOduration=2.148556058 podStartE2EDuration="2.148556058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:12:31.147171503 +0000 UTC m=+2515.604205960" watchObservedRunningTime="2026-07-28 12:12:31.148556058 +0000 UTC m=+2515.605590514"
Jul 28 12:12:32.042730 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:32.042695    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:32.047261 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:32.047235    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:32.133714 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:32.133684    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:32.134703 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:32.134685    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:38.361646 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.361608    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:12:38.362185 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.362164    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:38.362261 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.362188    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:38.362322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.362308    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a5481c-1634-4c12-abb3-f81ea8e3ef67" containerName="istio-proxy"
Jul 28 12:12:38.366803 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.366781    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.369783 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.369760    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-44546f-openshift-default-dockercfg-bnt76\""
Jul 28 12:12:38.380062 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.380035    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:12:38.438722 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438680    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438732    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438761    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrkkt\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438821    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438837    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438859    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.438922 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438879    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.439252 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438961    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.439252 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.438982    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540109 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540073    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540109 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540110    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540130    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrkkt\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540194    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540213    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540235    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540301    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540334    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540736 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540533    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540736 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540609    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540736 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540643    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.540939 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.540733    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.541149 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.541117    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.542738 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.542718    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.542991 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.542970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.547664 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.547643    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.547855 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.547837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrkkt\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt\") pod \"e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") " pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.677398 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.677317    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:38.815352 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.815326    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:12:38.817123 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:12:38.817091    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e881c1c_6065_47ff_9e91_38c6d7915706.slice/crio-520953dbe81a8c5e70852b89ef0007b6dec0c75a73f6efbea165dba4e97072cb WatchSource:0}: Error finding container 520953dbe81a8c5e70852b89ef0007b6dec0c75a73f6efbea165dba4e97072cb: Status 404 returned error can't find the container with id 520953dbe81a8c5e70852b89ef0007b6dec0c75a73f6efbea165dba4e97072cb
Jul 28 12:12:38.819646 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.819606    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:38.819752 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.819688    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:38.819752 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:38.819727    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:39.177606 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:39.177567    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerStarted","Data":"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"}
Jul 28 12:12:39.177825 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:39.177650    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerStarted","Data":"520953dbe81a8c5e70852b89ef0007b6dec0c75a73f6efbea165dba4e97072cb"}
Jul 28 12:12:39.677988 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:39.677949    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:39.683029 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:39.683002    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:39.703615 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:39.703574    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podStartSLOduration=1.7035582439999999 podStartE2EDuration="1.703558244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:12:39.19984868 +0000 UTC m=+2523.656883136" watchObservedRunningTime="2026-07-28 12:12:39.703558244 +0000 UTC m=+2524.160592700"
Jul 28 12:12:40.183290 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:40.183259    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:40.184249 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:40.184220    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:46.214724 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.214690    2552 generic.go:356] "Generic (PLEG): container finished" podID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerID="893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83" exitCode=0
Jul 28 12:12:46.215172 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.214744    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerDied","Data":"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"}
Jul 28 12:12:46.215172 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.215148    2552 scope.go:117] "RemoveContainer" containerID="893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"
Jul 28 12:12:46.216196 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.216167    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:46.216264 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.216239    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:46.216304 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:46.216275    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:47.134037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:47.133977    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:47.221312 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:47.221275    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerStarted","Data":"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"}
Jul 28 12:12:48.043231 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:48.042789    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:48.043231 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:48.042929    2552 patch_prober.go:28] interesting pod/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 28 12:12:48.043231 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:48.042971    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 28 12:12:49.042965 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.042927    2552 patch_prober.go:28] interesting pod/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 28 12:12:49.043541 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.043507    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 28 12:12:49.190727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.190695    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:12:49.195125 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.195099    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.197382 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.197362    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21898b\"/\"kube-root-ca.crt\""
Jul 28 12:12:49.198252 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.198224    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21898b\"/\"default-dockercfg-vrrtk\""
Jul 28 12:12:49.198347 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.198224    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21898b\"/\"test-model-21898b-kserve-self-signed-certs\""
Jul 28 12:12:49.198500 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.198475    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21898b\"/\"openshift-service-ca.crt\""
Jul 28 12:12:49.205480 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.205454    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244346    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244403    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244439    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244499    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgr2d\" (UniqueName: \"kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244534    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.253919 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.244572    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.345563 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.345466    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.345826 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.345806    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.345986 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.345965    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346180 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346161    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgr2d\" (UniqueName: \"kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346312 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346297    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346440 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346423    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346338    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346225    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.346973 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.346939    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.349744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.349722    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.353920 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.350353    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.358153 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.358125    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgr2d\" (UniqueName: \"kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d\") pod \"test-model-21898b-kserve-787c7fb969-pkgkw\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") " pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.505499 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.505455    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:49.697055 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:49.697028    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:12:49.699761 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:12:49.699728    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b9796e2_11c0_4940_a42e_dc131594a3bb.slice/crio-85a40f77d12d47e240431880c2dd9b62db540c1f9a6e7dd98aa668702a9bc464 WatchSource:0}: Error finding container 85a40f77d12d47e240431880c2dd9b62db540c1f9a6e7dd98aa668702a9bc464: Status 404 returned error can't find the container with id 85a40f77d12d47e240431880c2dd9b62db540c1f9a6e7dd98aa668702a9bc464
Jul 28 12:12:50.042668 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:50.042623    2552 patch_prober.go:28] interesting pod/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 28 12:12:50.042909 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:50.042699    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 28 12:12:50.243720 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:50.243673    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" event={"ID":"6b9796e2-11c0-4940-a42e-dc131594a3bb","Type":"ContainerStarted","Data":"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"}
Jul 28 12:12:50.244261 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:50.243749    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" event={"ID":"6b9796e2-11c0-4940-a42e-dc131594a3bb","Type":"ContainerStarted","Data":"85a40f77d12d47e240431880c2dd9b62db540c1f9a6e7dd98aa668702a9bc464"}
Jul 28 12:12:50.263558 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:50.263477    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" podStartSLOduration=1.263454229 podStartE2EDuration="1.263454229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:12:50.262264444 +0000 UTC m=+2534.719298901" watchObservedRunningTime="2026-07-28 12:12:50.263454229 +0000 UTC m=+2534.720488680"
Jul 28 12:12:51.042703 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:51.042652    2552 patch_prober.go:28] interesting pod/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 28 12:12:51.042953 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:51.042727    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 28 12:12:52.058010 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:52.057973    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:52.058510 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:52.058485    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:52.059241 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:52.059218    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:12:56.225116 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:56.225074    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 12:12:56.225589 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:56.225149    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 12:12:57.285102 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.285013    2552 generic.go:356] "Generic (PLEG): container finished" podID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerID="f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba" exitCode=0
Jul 28 12:12:57.285102 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.285075    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerDied","Data":"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"}
Jul 28 12:12:57.285536 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.285491    2552 scope.go:117] "RemoveContainer" containerID="f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"
Jul 28 12:12:57.286528 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.286497    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:57.286590 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.286570    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:57.286627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:57.286606    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:12:58.292323 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:58.292290    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerStarted","Data":"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"}
Jul 28 12:12:58.678611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:58.678511    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:12:58.678767 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:58.678729    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 12:12:58.678819 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:58.678772    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 12:12:59.505716 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:59.505669    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:59.506169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:59.505755    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:59.518432 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:59.518400    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:12:59.677992 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:59.677955    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 12:12:59.678169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:12:59.678018    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 12:13:00.312229 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:00.312190    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:13:00.678228 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:00.678138    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 12:13:00.678228 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:00.678199    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 12:13:01.678229 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:01.678191    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 12:13:01.678613 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:01.678245    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 12:13:02.681390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:02.681361    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:13:02.681754 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:02.681658    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:13:02.682443 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:13:02.682421    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:14:00.539681 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:00.539592    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:14:15.560063 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.560013    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" podUID="6b9796e2-11c0-4940-a42e-dc131594a3bb" containerName="main" containerID="cri-o://72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de" gracePeriod=45
Jul 28 12:14:15.806781 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.806756    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:14:15.844626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844550    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.844626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844614    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.844836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844658    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.844836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844697    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.844836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844722    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.844836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844778    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgr2d\" (UniqueName: \"kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d\") pod \"6b9796e2-11c0-4940-a42e-dc131594a3bb\" (UID: \"6b9796e2-11c0-4940-a42e-dc131594a3bb\") "
Jul 28 12:14:15.845093 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.844956    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:15.845093 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.845009    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home" (OuterVolumeSpecName: "home") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:15.845205 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.845104    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-model-cache\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:15.845205 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.845123    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-home\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:15.845353 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.845328    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:15.847106 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.847076    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm" (OuterVolumeSpecName: "dshm") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:15.847208 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.847156    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d" (OuterVolumeSpecName: "kube-api-access-xgr2d") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "kube-api-access-xgr2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:14:15.847346 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.847328    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b9796e2-11c0-4940-a42e-dc131594a3bb" (UID: "6b9796e2-11c0-4940-a42e-dc131594a3bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:14:15.945528 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.945497    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-tmp-dir\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:15.945528 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.945524    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b9796e2-11c0-4940-a42e-dc131594a3bb-dshm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:15.945712 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.945533    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgr2d\" (UniqueName: \"kubernetes.io/projected/6b9796e2-11c0-4940-a42e-dc131594a3bb-kube-api-access-xgr2d\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:15.945712 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:15.945552    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b9796e2-11c0-4940-a42e-dc131594a3bb-tls-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:16.661207 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.661167    2552 generic.go:356] "Generic (PLEG): container finished" podID="6b9796e2-11c0-4940-a42e-dc131594a3bb" containerID="72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de" exitCode=0
Jul 28 12:14:16.661626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.661239    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"
Jul 28 12:14:16.661626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.661253    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" event={"ID":"6b9796e2-11c0-4940-a42e-dc131594a3bb","Type":"ContainerDied","Data":"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"}
Jul 28 12:14:16.661626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.661305    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw" event={"ID":"6b9796e2-11c0-4940-a42e-dc131594a3bb","Type":"ContainerDied","Data":"85a40f77d12d47e240431880c2dd9b62db540c1f9a6e7dd98aa668702a9bc464"}
Jul 28 12:14:16.661626 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.661331    2552 scope.go:117] "RemoveContainer" containerID="72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"
Jul 28 12:14:16.669959 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.669922    2552 scope.go:117] "RemoveContainer" containerID="72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"
Jul 28 12:14:16.671952 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:14:16.671922    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de\": container with ID starting with 72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de not found: ID does not exist" containerID="72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"
Jul 28 12:14:16.672117 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.671962    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de"} err="failed to get container status \"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de\": rpc error: code = NotFound desc = could not find container \"72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de\": container with ID starting with 72ec650e0e8de9f6197885f5550c8fc60f8893e45ec18744d5135cea2a6146de not found: ID does not exist"
Jul 28 12:14:16.677289 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.677264    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:14:16.682501 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:16.682480    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-21898b/test-model-21898b-kserve-787c7fb969-pkgkw"]
Jul 28 12:14:18.109207 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:18.109176    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9796e2-11c0-4940-a42e-dc131594a3bb" path="/var/lib/kubelet/pods/6b9796e2-11c0-4940-a42e-dc131594a3bb/volumes"
Jul 28 12:14:23.702204 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:23.702167    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:14:23.702690 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:23.702387    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy" containerID="cri-o://879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd" gracePeriod=30
Jul 28 12:14:28.960377 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:28.960352    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:14:29.068561 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068464    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7smtx\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068561 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068514    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068561 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068537    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068571    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068622    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068746    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068773    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068795    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.068839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068830    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket\") pod \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\" (UID: \"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f\") "
Jul 28 12:14:29.069192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.068862    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:29.069192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.069116    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:29.069192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.069143    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.069192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.069178    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:14:29.069409 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.069337    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:29.069579 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.069540    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:29.070988 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.070962    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:14:29.071087 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.071035    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:29.071131 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.071085    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:14:29.071131 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.071108    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx" (OuterVolumeSpecName: "kube-api-access-7smtx") pod "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" (UID: "07f05b2e-f3a5-4a02-a28c-f0b13e141a8f"). InnerVolumeSpecName "kube-api-access-7smtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:14:29.170235 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170191    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170235 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170226    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170235 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170240    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170253    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170267    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7smtx\" (UniqueName: \"kubernetes.io/projected/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-kube-api-access-7smtx\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170280    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170292    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.170526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.170304    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:29.725389 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.725356    2552 generic.go:356] "Generic (PLEG): container finished" podID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerID="879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd" exitCode=0
Jul 28 12:14:29.725572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.725402    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"
Jul 28 12:14:29.725572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.725425    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerDied","Data":"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"}
Jul 28 12:14:29.725572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.725468    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr" event={"ID":"07f05b2e-f3a5-4a02-a28c-f0b13e141a8f","Type":"ContainerDied","Data":"d0fdcbc1aff2ec8fc01607c51fc00863e3e2e2b74387bea6e60636cb8d267c26"}
Jul 28 12:14:29.725572 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.725491    2552 scope.go:117] "RemoveContainer" containerID="879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"
Jul 28 12:14:29.735328 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.735308    2552 scope.go:117] "RemoveContainer" containerID="893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"
Jul 28 12:14:29.742887 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.742868    2552 scope.go:117] "RemoveContainer" containerID="879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"
Jul 28 12:14:29.743174 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:14:29.743151    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd\": container with ID starting with 879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd not found: ID does not exist" containerID="879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"
Jul 28 12:14:29.743219 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.743181    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd"} err="failed to get container status \"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd\": rpc error: code = NotFound desc = could not find container \"879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd\": container with ID starting with 879ae782601ecbb425baef212e0c54115d0426d8780284cdfb7f12bd971611cd not found: ID does not exist"
Jul 28 12:14:29.743219 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.743199    2552 scope.go:117] "RemoveContainer" containerID="893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"
Jul 28 12:14:29.743466 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:14:29.743448    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83\": container with ID starting with 893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83 not found: ID does not exist" containerID="893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"
Jul 28 12:14:29.743506 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.743473    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83"} err="failed to get container status \"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83\": rpc error: code = NotFound desc = could not find container \"893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83\": container with ID starting with 893c7e5084ac1d2b208f16362062b2e1eadf0bdd4f68e8d08a32ccbee3420d83 not found: ID does not exist"
Jul 28 12:14:29.747813 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.747791    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:14:29.751288 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:29.751268    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-21898b-openshift-default-659677d958-z7lmr"]
Jul 28 12:14:30.108839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:30.108760    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" path="/var/lib/kubelet/pods/07f05b2e-f3a5-4a02-a28c-f0b13e141a8f/volumes"
Jul 28 12:14:46.035018 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:46.034980    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:14:46.035549 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:46.035196    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" containerID="cri-o://9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5" gracePeriod=30
Jul 28 12:14:47.024674 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.024639    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:14:47.025146 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025126    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.025146 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025148    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025164    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025172    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025212    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9796e2-11c0-4940-a42e-dc131594a3bb" containerName="main"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025221    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9796e2-11c0-4940-a42e-dc131594a3bb" containerName="main"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025310    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.025320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025321    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9796e2-11c0-4940-a42e-dc131594a3bb" containerName="main"
Jul 28 12:14:47.025608 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.025335    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f05b2e-f3a5-4a02-a28c-f0b13e141a8f" containerName="istio-proxy"
Jul 28 12:14:47.028443 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.028421    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.032065 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.032044    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-560abc-openshift-default-dockercfg-hltg6\""
Jul 28 12:14:47.041693 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.041668    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:14:47.128975 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.128922    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129158 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.128988    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129158 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129032    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129158 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129059    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129158 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129144    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129358 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129172    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129358 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129197    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqsl\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129358 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129244    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.129358 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.129279    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.229937 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.229902    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.229957    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.229978    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.229993    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqsl\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230010    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230029    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230128    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230483 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230483 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230199    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230483 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230640 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230497    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230640 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230566    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230710    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.230857 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.230835    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.232355 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.232327    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.232562 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.232545    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.238138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.238117    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqsl\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.238390 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.238370    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token\") pod \"e2e-api-a-560abc-openshift-default-bf658fd94-jdndl\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") " pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.338653 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.338565    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:47.472101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.472065    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:14:47.475128 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:14:47.475102    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52dcd1cb_2c33_4a6d_a292_26f2f7db949b.slice/crio-01fbd29c52ef711832ab3f52d60600d86d6c88e0809f2b85cde8a98d2b1319a5 WatchSource:0}: Error finding container 01fbd29c52ef711832ab3f52d60600d86d6c88e0809f2b85cde8a98d2b1319a5: Status 404 returned error can't find the container with id 01fbd29c52ef711832ab3f52d60600d86d6c88e0809f2b85cde8a98d2b1319a5
Jul 28 12:14:47.477259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.477222    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:47.477339 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.477299    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:47.477339 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.477327    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:47.684684 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.684605    2552 patch_prober.go:28] interesting pod/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 12:14:47.684874 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.684675    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 12:14:47.812324 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.812284    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerStarted","Data":"754facfff1efac2b6e339ad4b6ed50041643234d252719bc748d328c1ff4bae0"}
Jul 28 12:14:47.812526 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.812346    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerStarted","Data":"01fbd29c52ef711832ab3f52d60600d86d6c88e0809f2b85cde8a98d2b1319a5"}
Jul 28 12:14:47.831598 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:47.831549    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podStartSLOduration=0.831529428 podStartE2EDuration="831.529428ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:14:47.829886581 +0000 UTC m=+2652.286921038" watchObservedRunningTime="2026-07-28 12:14:47.831529428 +0000 UTC m=+2652.288563886"
Jul 28 12:14:48.338861 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:48.338819    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:48.343928 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:48.343884    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:48.816965 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:48.816938    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:48.817883 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:48.817865    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:14:51.287782 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.287750    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:14:51.367353 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367253    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367353 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367307    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367353 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367329    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367353 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367352    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367397    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367427    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367458    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367496    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrkkt\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367530    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data\") pod \"4e881c1c-6065-47ff-9e91-38c6d7915706\" (UID: \"4e881c1c-6065-47ff-9e91-38c6d7915706\") "
Jul 28 12:14:51.367700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367536    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:51.368037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367740    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:51.368037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367768    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:51.368037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367785    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.368037 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.367809    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:14:51.368630 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.368604    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:51.369857 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.369828    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:14:51.369857 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.369838    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt" (OuterVolumeSpecName: "kube-api-access-vrkkt") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "kube-api-access-vrkkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:14:51.370085 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.370064    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:14:51.370150 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.370070    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e881c1c-6065-47ff-9e91-38c6d7915706" (UID: "4e881c1c-6065-47ff-9e91-38c6d7915706"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:14:51.468913 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468868    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e881c1c-6065-47ff-9e91-38c6d7915706-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.468913 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468916    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468926    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468937    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468946    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468955    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468962    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrkkt\" (UniqueName: \"kubernetes.io/projected/4e881c1c-6065-47ff-9e91-38c6d7915706-kube-api-access-vrkkt\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.469118 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.468971    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e881c1c-6065-47ff-9e91-38c6d7915706-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:14:51.832251 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.832214    2552 generic.go:356] "Generic (PLEG): container finished" podID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerID="9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5" exitCode=0
Jul 28 12:14:51.832450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.832259    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerDied","Data":"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"}
Jul 28 12:14:51.832450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.832273    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"
Jul 28 12:14:51.832450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.832347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn" event={"ID":"4e881c1c-6065-47ff-9e91-38c6d7915706","Type":"ContainerDied","Data":"520953dbe81a8c5e70852b89ef0007b6dec0c75a73f6efbea165dba4e97072cb"}
Jul 28 12:14:51.832450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.832370    2552 scope.go:117] "RemoveContainer" containerID="9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"
Jul 28 12:14:51.841535 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.841510    2552 scope.go:117] "RemoveContainer" containerID="f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"
Jul 28 12:14:51.849394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.849373    2552 scope.go:117] "RemoveContainer" containerID="9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"
Jul 28 12:14:51.849645 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:14:51.849624    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5\": container with ID starting with 9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5 not found: ID does not exist" containerID="9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"
Jul 28 12:14:51.849721 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.849669    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5"} err="failed to get container status \"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5\": rpc error: code = NotFound desc = could not find container \"9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5\": container with ID starting with 9e3524ad94788621d9c9c572f58ff260a9f63fbc630f388b86a8684bcb91dfe5 not found: ID does not exist"
Jul 28 12:14:51.849721 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.849696    2552 scope.go:117] "RemoveContainer" containerID="f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"
Jul 28 12:14:51.849997 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:14:51.849975    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba\": container with ID starting with f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba not found: ID does not exist" containerID="f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"
Jul 28 12:14:51.850061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.850003    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba"} err="failed to get container status \"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba\": rpc error: code = NotFound desc = could not find container \"f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba\": container with ID starting with f30e12aad3d6b6c4b9163f16d5525cdcba82818d878c5249cf4447b6c64ad2ba not found: ID does not exist"
Jul 28 12:14:51.855831 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.855809    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:14:51.860079 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:51.860057    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-44546f-openshift-default-987d8877d-xhnhn"]
Jul 28 12:14:52.112978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:52.110793    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" path="/var/lib/kubelet/pods/4e881c1c-6065-47ff-9e91-38c6d7915706/volumes"
Jul 28 12:14:55.203418 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203383    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:14:55.203822 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203745    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.203822 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203756    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.203822 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203770    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.203822 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203775    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.204011 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203846    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.204011 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.203856    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e881c1c-6065-47ff-9e91-38c6d7915706" containerName="istio-proxy"
Jul 28 12:14:55.208760 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.208735    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.210650 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.210628    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-88898f-openshift-default-dockercfg-6zld9\""
Jul 28 12:14:55.218027 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.218004    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:14:55.303343 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303278    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303343 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303330    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5xp\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303390    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303441    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303460    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303547    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303794 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303614    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303794 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303665    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.303794 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.303693    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404388 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404351    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404415    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404442    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404476    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404515    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404560    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404609 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404633    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404663    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5xp\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404890    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.404985 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.404952    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.405169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.405044    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.405169 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.405052    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.405357 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.405337    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.406830 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.406801    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.407083 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.407062    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.412807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.412778    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5xp\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.412807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.412716    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token\") pod \"e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") " pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.520818 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.520744    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:55.660962 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.660935    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:14:55.663617 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:14:55.663591    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27da172_06bd_4e22_9e19_b30a189ea83e.slice/crio-05ea8930078d316eba5cbbbc416562d5d1fb99acb9d2e64af31750722f91a538 WatchSource:0}: Error finding container 05ea8930078d316eba5cbbbc416562d5d1fb99acb9d2e64af31750722f91a538: Status 404 returned error can't find the container with id 05ea8930078d316eba5cbbbc416562d5d1fb99acb9d2e64af31750722f91a538
Jul 28 12:14:55.665836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.665795    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:55.665970 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.665881    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:55.665970 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.665942    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:14:55.853422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.853331    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerStarted","Data":"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"}
Jul 28 12:14:55.853422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.853386    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerStarted","Data":"05ea8930078d316eba5cbbbc416562d5d1fb99acb9d2e64af31750722f91a538"}
Jul 28 12:14:55.902442 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:55.902390    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podStartSLOduration=0.902373266 podStartE2EDuration="902.373266ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:14:55.902203086 +0000 UTC m=+2660.359237546" watchObservedRunningTime="2026-07-28 12:14:55.902373266 +0000 UTC m=+2660.359407722"
Jul 28 12:14:56.521138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:56.521099    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:56.526042 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:56.526017    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:56.861301 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:56.861215    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:14:56.862121 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:14:56.862100    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:15:04.859090 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:04.859041    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 12:15:04.859611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:04.859114    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 12:15:06.908224 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.908134    2552 generic.go:356] "Generic (PLEG): container finished" podID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerID="754facfff1efac2b6e339ad4b6ed50041643234d252719bc748d328c1ff4bae0" exitCode=0
Jul 28 12:15:06.908224 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.908185    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerDied","Data":"754facfff1efac2b6e339ad4b6ed50041643234d252719bc748d328c1ff4bae0"}
Jul 28 12:15:06.908679 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.908569    2552 scope.go:117] "RemoveContainer" containerID="754facfff1efac2b6e339ad4b6ed50041643234d252719bc748d328c1ff4bae0"
Jul 28 12:15:06.909611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.909580    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:06.909671 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.909658    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:06.909712 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:06.909691    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:07.915534 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:07.915498    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerStarted","Data":"1f2c144fc66e23fa994b6e36b35c2096f471b376f889343fe8c635d9bcba01bc"}
Jul 28 12:15:08.339321 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:08.339281    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:08.339880 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:08.339784    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 12:15:08.340123 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:08.340095    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 12:15:09.339317 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:09.339280    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 12:15:09.339959 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:09.339924    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 12:15:10.339175 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:10.339122    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 12:15:10.339420 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:10.339206    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 12:15:11.339998 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:11.339957    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 12:15:11.340573 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:11.340539    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 12:15:12.343294 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:12.343252    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:12.343789 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:12.343768    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:12.344772 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:12.344747    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:12.903051 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:12.903006    2552 patch_prober.go:28] interesting pod/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 12:15:12.903225 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:12.903074    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 12:15:13.945547 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.945522    2552 generic.go:356] "Generic (PLEG): container finished" podID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerID="3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4" exitCode=0
Jul 28 12:15:13.945929 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.945597    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerDied","Data":"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"}
Jul 28 12:15:13.946185 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.946168    2552 scope.go:117] "RemoveContainer" containerID="3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"
Jul 28 12:15:13.947324 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.947297    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:13.947384 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.947371    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:13.947423 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:13.947411    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:15:14.955727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:14.955670    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerStarted","Data":"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"}
Jul 28 12:15:15.521121 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:15.521080    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:15:15.521344 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:15.521321    2552 patch_prober.go:28] interesting pod/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 12:15:15.521410 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:15.521384    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 12:15:16.522151 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:16.522113    2552 patch_prober.go:28] interesting pod/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 12:15:16.522581 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:16.522169    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 12:15:17.522073 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:17.522035    2552 patch_prober.go:28] interesting pod/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 12:15:17.522260 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:17.522102    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 12:15:18.522177 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:18.522131    2552 patch_prober.go:28] interesting pod/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 12:15:18.522398 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:18.522199    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 12:15:19.526041 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:19.526005    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:15:19.526418 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:19.526262    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:15:19.527027 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:19.527009    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:15:37.913704 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:37.913664    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:15:37.914218 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:37.913916    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" containerID="cri-o://1f2c144fc66e23fa994b6e36b35c2096f471b376f889343fe8c635d9bcba01bc" gracePeriod=30
Jul 28 12:15:42.346138 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:42.346096    2552 patch_prober.go:28] interesting pod/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 12:15:42.346527 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:42.346158    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 12:15:43.094968 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.094937    2552 generic.go:356] "Generic (PLEG): container finished" podID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerID="1f2c144fc66e23fa994b6e36b35c2096f471b376f889343fe8c635d9bcba01bc" exitCode=0
Jul 28 12:15:43.095156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.095032    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerDied","Data":"1f2c144fc66e23fa994b6e36b35c2096f471b376f889343fe8c635d9bcba01bc"}
Jul 28 12:15:43.095156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.095125    2552 scope.go:117] "RemoveContainer" containerID="754facfff1efac2b6e339ad4b6ed50041643234d252719bc748d328c1ff4bae0"
Jul 28 12:15:43.171790 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.171764    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:43.254055 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254023    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254055 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254060    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254292 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254088    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254292 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254123    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254292 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254149    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254292 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254179    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254292 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254239    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254551 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254295    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254551 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254323    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:15:43.254551 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254344    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qqsl\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl\") pod \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\" (UID: \"52dcd1cb-2c33-4a6d-a292-26f2f7db949b\") "
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254547    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254570    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254590    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254677    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254692    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254701    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.254731 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.254714    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.255302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.255275    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data" (OuterVolumeSpecName: "istio-data") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:15:43.256434 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.256404    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token" (OuterVolumeSpecName: "istio-token") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:15:43.256537 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.256520    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:15:43.256807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.256784    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl" (OuterVolumeSpecName: "kube-api-access-8qqsl") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "kube-api-access-8qqsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:15:43.256807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.256785    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52dcd1cb-2c33-4a6d-a292-26f2f7db949b" (UID: "52dcd1cb-2c33-4a6d-a292-26f2f7db949b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:15:43.355241 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.355211    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.355241 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.355238    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.355241 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.355250    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.355674 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.355262    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:43.355674 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:43.355273    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qqsl\" (UniqueName: \"kubernetes.io/projected/52dcd1cb-2c33-4a6d-a292-26f2f7db949b-kube-api-access-8qqsl\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:15:44.101245 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:44.101206    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl" event={"ID":"52dcd1cb-2c33-4a6d-a292-26f2f7db949b","Type":"ContainerDied","Data":"01fbd29c52ef711832ab3f52d60600d86d6c88e0809f2b85cde8a98d2b1319a5"}
Jul 28 12:15:44.101245 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:44.101235    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"
Jul 28 12:15:44.101506 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:44.101277    2552 scope.go:117] "RemoveContainer" containerID="1f2c144fc66e23fa994b6e36b35c2096f471b376f889343fe8c635d9bcba01bc"
Jul 28 12:15:44.132360 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:44.132329    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:15:44.139906 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:44.139868    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-560abc-openshift-default-bf658fd94-jdndl"]
Jul 28 12:15:46.109402 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:15:46.109365    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" path="/var/lib/kubelet/pods/52dcd1cb-2c33-4a6d-a292-26f2f7db949b/volumes"
Jul 28 12:16:05.227624 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:05.227590    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:16:05.228103 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:05.227819    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy" containerID="cri-o://e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801" gracePeriod=30
Jul 28 12:16:06.183180 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.183148    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:16:06.183566 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.183540    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:06.183566 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.183563    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:06.183770 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.183672    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:06.183770 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.183683    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:06.186957 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.186938    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.188998 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.188979    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4d5cfe-openshift-default-dockercfg-lnz5m\""
Jul 28 12:16:06.202484 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.202462    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:16:06.261913 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.261868    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.261922    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9r2q\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.261945    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262030    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262073    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262097    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262169    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262193    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.262322 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.262275    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363057 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363018    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363084    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363113    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9r2q\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363139    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363179    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363209    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363234    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363559 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363280    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363559 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363308    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363559 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363419    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363559 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363483    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363843 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363556    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.363843 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.363599    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.364093 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.364073    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.365485 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.365459    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.365607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.365588    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.370558 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.370531    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.370774 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.370754    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9r2q\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q\") pod \"e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") " pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.496984 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.496935    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:06.633384 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.633359    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:16:06.635507 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:16:06.635478    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d317fd_174f_4422_945c_7fb821a9d08e.slice/crio-b3be519140cd11be753d2aea611df1be9cf90d65d05a74edb7899f5c2a15af5d WatchSource:0}: Error finding container b3be519140cd11be753d2aea611df1be9cf90d65d05a74edb7899f5c2a15af5d: Status 404 returned error can't find the container with id b3be519140cd11be753d2aea611df1be9cf90d65d05a74edb7899f5c2a15af5d
Jul 28 12:16:06.637613 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.637578    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:06.637697 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.637646    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:06.637697 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:06.637675    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:07.215771 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:07.215735    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerStarted","Data":"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"}
Jul 28 12:16:07.216013 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:07.215799    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerStarted","Data":"b3be519140cd11be753d2aea611df1be9cf90d65d05a74edb7899f5c2a15af5d"}
Jul 28 12:16:07.237092 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:07.237044    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podStartSLOduration=1.237027915 podStartE2EDuration="1.237027915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:16:07.236066002 +0000 UTC m=+2731.693100473" watchObservedRunningTime="2026-07-28 12:16:07.237027915 +0000 UTC m=+2731.694062371"
Jul 28 12:16:07.497653 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:07.497563    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:07.502177 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:07.502151    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:08.220157 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:08.220119    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:08.221032 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:08.221012    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:10.485488 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.485458    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:16:10.604944 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.604834    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.604944 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.604876    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.604944 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.604928    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.604972    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.604988    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605082    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605143    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb5xp\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605190    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605214    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket\") pod \"f27da172-06bd-4e22-9e19-b30a189ea83e\" (UID: \"f27da172-06bd-4e22-9e19-b30a189ea83e\") "
Jul 28 12:16:10.605243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605213    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:16:10.605619 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605325    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:16:10.605619 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605408    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:16:10.605619 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605551    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.605619 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605569    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.605619 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605586    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27da172-06bd-4e22-9e19-b30a189ea83e-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.605891 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.605709    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:16:10.606318 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.606288    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data" (OuterVolumeSpecName: "istio-data") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:16:10.607366 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.607344    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:16:10.607366 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.607350    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp" (OuterVolumeSpecName: "kube-api-access-kb5xp") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "kube-api-access-kb5xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:16:10.607468 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.607360    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:16:10.607468 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.607458    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token" (OuterVolumeSpecName: "istio-token") pod "f27da172-06bd-4e22-9e19-b30a189ea83e" (UID: "f27da172-06bd-4e22-9e19-b30a189ea83e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:16:10.706280 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706245    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.706280 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706275    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.706477 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706290    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.706477 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706303    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb5xp\" (UniqueName: \"kubernetes.io/projected/f27da172-06bd-4e22-9e19-b30a189ea83e-kube-api-access-kb5xp\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.706477 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706318    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:10.706477 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:10.706330    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27da172-06bd-4e22-9e19-b30a189ea83e-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:16:11.236810 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.236778    2552 generic.go:356] "Generic (PLEG): container finished" podID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerID="e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801" exitCode=0
Jul 28 12:16:11.237005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.236822    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"
Jul 28 12:16:11.237005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.236860    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerDied","Data":"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"}
Jul 28 12:16:11.237005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.236938    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r" event={"ID":"f27da172-06bd-4e22-9e19-b30a189ea83e","Type":"ContainerDied","Data":"05ea8930078d316eba5cbbbc416562d5d1fb99acb9d2e64af31750722f91a538"}
Jul 28 12:16:11.237005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.236970    2552 scope.go:117] "RemoveContainer" containerID="e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"
Jul 28 12:16:11.246887 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.246869    2552 scope.go:117] "RemoveContainer" containerID="3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"
Jul 28 12:16:11.254610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.254593    2552 scope.go:117] "RemoveContainer" containerID="e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"
Jul 28 12:16:11.254882 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:16:11.254863    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801\": container with ID starting with e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801 not found: ID does not exist" containerID="e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"
Jul 28 12:16:11.254990 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.254911    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801"} err="failed to get container status \"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801\": rpc error: code = NotFound desc = could not find container \"e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801\": container with ID starting with e068c8f2f382004b56d19226a41b9d3891c9bacfd8cf037e7abb9d06bf73f801 not found: ID does not exist"
Jul 28 12:16:11.254990 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.254937    2552 scope.go:117] "RemoveContainer" containerID="3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"
Jul 28 12:16:11.255220 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:16:11.255203    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4\": container with ID starting with 3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4 not found: ID does not exist" containerID="3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"
Jul 28 12:16:11.255266 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.255226    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4"} err="failed to get container status \"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4\": rpc error: code = NotFound desc = could not find container \"3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4\": container with ID starting with 3c73fcf263edd8cbab978f10ab8b7041c937107d7281772d9a0597e4a16832f4 not found: ID does not exist"
Jul 28 12:16:11.260487 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.260460    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:16:11.263726 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:11.263706    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-88898f-openshift-default-6fd665dcb4-f9f9r"]
Jul 28 12:16:12.108224 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:12.108188    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" path="/var/lib/kubelet/pods/f27da172-06bd-4e22-9e19-b30a189ea83e/volumes"
Jul 28 12:16:14.404311 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404281    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404656    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404667    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404679    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404686    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="52dcd1cb-2c33-4a6d-a292-26f2f7db949b" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404696    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404702    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:16:14.404809 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.404757    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:16:14.408869 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.408848    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.411045 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.411023    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e7af0c-openshift-default-dockercfg-jbnnv\""
Jul 28 12:16:14.423498 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.423476    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:16:14.541693 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541652    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.541862 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541701    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.541862 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541752    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.541862 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541773    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqt22\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.541862 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541792    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.542034 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.541944    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.542034 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.542013    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.542101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.542051    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.542101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.542075    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.642825 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642786    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.642825 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642827    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642847    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642868    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqt22\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642914    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.642966    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643002    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643034    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643101 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643480 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643288    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643480 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643304    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643579 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643511    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643579 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643536    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.643693 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.643646    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.645479 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.645455    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.645551 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.645506    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.656219 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.656150    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqt22\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.658302 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.658275    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token\") pod \"e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") " pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.719807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.719772    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:14.860633 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.860592    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:16:14.862875 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:16:14.862848    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215971c1_dd5b_4944_ad59_9220a7bb8974.slice/crio-e8683fb7bd4c66e2fc82e80b55a6d1b242871387c32b1239cda9cdf7b150fea2 WatchSource:0}: Error finding container e8683fb7bd4c66e2fc82e80b55a6d1b242871387c32b1239cda9cdf7b150fea2: Status 404 returned error can't find the container with id e8683fb7bd4c66e2fc82e80b55a6d1b242871387c32b1239cda9cdf7b150fea2
Jul 28 12:16:14.865058 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.865023    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:14.865117 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.865093    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:14.865158 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:14.865123    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:15.258427 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:15.258391    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerStarted","Data":"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"}
Jul 28 12:16:15.258594 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:15.258444    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerStarted","Data":"e8683fb7bd4c66e2fc82e80b55a6d1b242871387c32b1239cda9cdf7b150fea2"}
Jul 28 12:16:15.279320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:15.279272    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podStartSLOduration=1.279258676 podStartE2EDuration="1.279258676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:16:15.277735409 +0000 UTC m=+2739.734769876" watchObservedRunningTime="2026-07-28 12:16:15.279258676 +0000 UTC m=+2739.736293134"
Jul 28 12:16:15.720976 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:15.720941    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:15.725537 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:15.725513    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:16.263185 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:16.263154    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:16.264191 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:16.264167    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:24.262083 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:24.262030    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 12:16:24.262490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:24.262103    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 12:16:25.307981 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.307942    2552 generic.go:356] "Generic (PLEG): container finished" podID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerID="0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd" exitCode=0
Jul 28 12:16:25.308404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.308015    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerDied","Data":"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"}
Jul 28 12:16:25.308404 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.308386    2552 scope.go:117] "RemoveContainer" containerID="0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"
Jul 28 12:16:25.309567 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.309538    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:25.309637 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.309621    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:25.309678 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:25.309665    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:26.314857 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:26.314814    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerStarted","Data":"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"}
Jul 28 12:16:26.498023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:26.497979    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:26.498248 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:26.498095    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 12:16:26.498248 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:26.498144    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 12:16:27.498243 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:27.498200    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 12:16:27.498645 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:27.498262    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 12:16:28.497977 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:28.497940    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 12:16:28.498156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:28.497999    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 12:16:29.497879 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:29.497836    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 12:16:29.498278 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:29.497957    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 12:16:30.497723 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:30.497686    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 12:16:30.497924 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:30.497743    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 12:16:31.501242 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:31.501202    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:31.501682 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:31.501477    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:31.502246 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:31.502226    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:16:31.833611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:31.832723    2552 patch_prober.go:28] interesting pod/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:50248->10.132.0.47:15021: read: connection reset by peer" start-of-body=
Jul 28 12:16:31.833611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:31.832792    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": read tcp 10.132.0.2:50248->10.132.0.47:15021: read: connection reset by peer"
Jul 28 12:16:32.348214 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.348176    2552 generic.go:356] "Generic (PLEG): container finished" podID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerID="6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87" exitCode=0
Jul 28 12:16:32.348395 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.348253    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerDied","Data":"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"}
Jul 28 12:16:32.348738 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.348719    2552 scope.go:117] "RemoveContainer" containerID="6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"
Jul 28 12:16:32.349701 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.349676    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:32.349753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.349742    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:32.349789 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:32.349774    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:16:33.358287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:33.358247    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerStarted","Data":"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"}
Jul 28 12:16:33.720288 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:33.720253    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:33.720476 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:33.720451    2552 patch_prober.go:28] interesting pod/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 12:16:33.720542 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:33.720516    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 12:16:34.720395 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:34.720360    2552 patch_prober.go:28] interesting pod/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 12:16:34.720785 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:34.720428    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 12:16:35.720686 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:35.720650    2552 patch_prober.go:28] interesting pod/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 12:16:35.721131 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:35.720709    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 12:16:36.720228 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:36.720194    2552 patch_prober.go:28] interesting pod/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 12:16:36.720415 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:36.720255    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 12:16:37.723955 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:37.723921    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:37.724426 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:37.724221    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:16:37.725002 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:16:37.724983    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:17:04.110356 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.110268    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:17:04.110918 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.110870    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27da172-06bd-4e22-9e19-b30a189ea83e" containerName="istio-proxy"
Jul 28 12:17:04.114079 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.114057    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.117040 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.117014    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"openshift-service-ca.crt\""
Jul 28 12:17:04.117163 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.117056    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"auth-test-model-e7af0c-kserve-self-signed-certs\""
Jul 28 12:17:04.117163 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.117014    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"default-dockercfg-sn6d8\""
Jul 28 12:17:04.117163 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.117107    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"kube-root-ca.crt\""
Jul 28 12:17:04.121457 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.121434    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:17:04.209519 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209487    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.209689 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8kq\" (UniqueName: \"kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.209689 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209625    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.209689 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209663    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.209807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209726    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.209807 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.209774    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310652 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310677    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310718    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8kq\" (UniqueName: \"kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310785    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310810    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.310868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.310840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.311239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.311209    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.311441 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.311416    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.311441 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.311439    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.313188 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.313163    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.313412 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.313392    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.319094 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.319076    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8kq\" (UniqueName: \"kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq\") pod \"auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") " pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.424343 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.424242    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:04.557864 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:04.557828    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:17:04.562161 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:17:04.562131    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26052ac7_68a8_45cf_b5f7_78d619964ec4.slice/crio-f15a8bf028559df7cb3cfa021226b5805480b3c4463dcc125dc8b5ea3f44ef3e WatchSource:0}: Error finding container f15a8bf028559df7cb3cfa021226b5805480b3c4463dcc125dc8b5ea3f44ef3e: Status 404 returned error can't find the container with id f15a8bf028559df7cb3cfa021226b5805480b3c4463dcc125dc8b5ea3f44ef3e
Jul 28 12:17:05.516540 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:05.516504    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" event={"ID":"26052ac7-68a8-45cf-b5f7-78d619964ec4","Type":"ContainerStarted","Data":"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"}
Jul 28 12:17:05.517002 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:05.516548    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" event={"ID":"26052ac7-68a8-45cf-b5f7-78d619964ec4","Type":"ContainerStarted","Data":"f15a8bf028559df7cb3cfa021226b5805480b3c4463dcc125dc8b5ea3f44ef3e"}
Jul 28 12:17:05.533875 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:05.533823    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" podStartSLOduration=1.53379859 podStartE2EDuration="1.53379859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:17:05.532757122 +0000 UTC m=+2789.989791579" watchObservedRunningTime="2026-07-28 12:17:05.53379859 +0000 UTC m=+2789.990833047"
Jul 28 12:17:14.424555 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:14.424513    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:14.425028 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:14.424571    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:14.436783 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:14.436757    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:17:14.569506 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:17:14.569473    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:21:16.110719 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:16.110688    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:21:28.389438 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.389351    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:21:28.392797 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.392778    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.394861 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.394836    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d5cfe\"/\"openshift-service-ca.crt\""
Jul 28 12:21:28.395095 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.395078    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d5cfe\"/\"kube-root-ca.crt\""
Jul 28 12:21:28.395645 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.395624    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d5cfe\"/\"default-dockercfg-8gj6v\""
Jul 28 12:21:28.395748 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.395650    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d5cfe\"/\"sub-test-model-4d5cfe-kserve-self-signed-certs\""
Jul 28 12:21:28.405627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.405594    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:21:28.465786 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465743    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.465978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.465978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465843    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.465978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465875    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.465978 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465954    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhnc2\" (UniqueName: \"kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.466127 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.465991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.566801 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.566762    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567025 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.566860    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567025 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.566918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567025 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.566954    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567025 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.566987    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567256 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.567039    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhnc2\" (UniqueName: \"kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567440 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.567404    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.567440    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.567554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.567451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.569260 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.569233    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.569520 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.569497    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.574653 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.574631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhnc2\" (UniqueName: \"kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2\") pod \"sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") " pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.705074 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.705045    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:28.834573 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:28.834544    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:21:28.837265 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:21:28.837235    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78262aab_7b84_416f_b616_1e1edf64a805.slice/crio-86d805653c9c28c92b8b56132cad2ffb96805ec11ab476aef6d47a5431190818 WatchSource:0}: Error finding container 86d805653c9c28c92b8b56132cad2ffb96805ec11ab476aef6d47a5431190818: Status 404 returned error can't find the container with id 86d805653c9c28c92b8b56132cad2ffb96805ec11ab476aef6d47a5431190818
Jul 28 12:21:29.769180 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:29.769141    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" event={"ID":"78262aab-7b84-416f-b616-1e1edf64a805","Type":"ContainerStarted","Data":"89760a688fe054650057a1506fff28edffeac4d3dd1afcbb68730dcdc9fa1b81"}
Jul 28 12:21:29.769602 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:29.769191    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" event={"ID":"78262aab-7b84-416f-b616-1e1edf64a805","Type":"ContainerStarted","Data":"86d805653c9c28c92b8b56132cad2ffb96805ec11ab476aef6d47a5431190818"}
Jul 28 12:21:29.784864 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:29.784802    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" podStartSLOduration=1.784785636 podStartE2EDuration="1.784785636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:21:29.783763257 +0000 UTC m=+3054.240797713" watchObservedRunningTime="2026-07-28 12:21:29.784785636 +0000 UTC m=+3054.241820093"
Jul 28 12:21:31.129354 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.129313    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" podUID="26052ac7-68a8-45cf-b5f7-78d619964ec4" containerName="main" containerID="cri-o://34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113" gracePeriod=45
Jul 28 12:21:31.384536 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.384478    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:21:31.496364 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496332    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496550 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496403    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496550 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496440    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs8kq\" (UniqueName: \"kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496550 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496514    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496550 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496534    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496801 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496553    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home\") pod \"26052ac7-68a8-45cf-b5f7-78d619964ec4\" (UID: \"26052ac7-68a8-45cf-b5f7-78d619964ec4\") "
Jul 28 12:21:31.496801 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496668    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:21:31.497250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496800    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache" (OuterVolumeSpecName: "model-cache") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:21:31.497250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.496868    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-tmp-dir\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.497250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.497104    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home" (OuterVolumeSpecName: "home") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:21:31.498745 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.498719    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm" (OuterVolumeSpecName: "dshm") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:21:31.498863 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.498820    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:21:31.498939 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.498923    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq" (OuterVolumeSpecName: "kube-api-access-xs8kq") pod "26052ac7-68a8-45cf-b5f7-78d619964ec4" (UID: "26052ac7-68a8-45cf-b5f7-78d619964ec4"). InnerVolumeSpecName "kube-api-access-xs8kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:21:31.598434 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.598392    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-dshm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.598434 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.598431    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs8kq\" (UniqueName: \"kubernetes.io/projected/26052ac7-68a8-45cf-b5f7-78d619964ec4-kube-api-access-xs8kq\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.598623 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.598445    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-model-cache\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.598623 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.598460    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26052ac7-68a8-45cf-b5f7-78d619964ec4-tls-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.598623 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.598471    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26052ac7-68a8-45cf-b5f7-78d619964ec4-home\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:21:31.779867 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.779836    2552 generic.go:356] "Generic (PLEG): container finished" podID="26052ac7-68a8-45cf-b5f7-78d619964ec4" containerID="34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113" exitCode=0
Jul 28 12:21:31.780061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.779878    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"
Jul 28 12:21:31.780061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.779889    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" event={"ID":"26052ac7-68a8-45cf-b5f7-78d619964ec4","Type":"ContainerDied","Data":"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"}
Jul 28 12:21:31.780061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.779943    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk" event={"ID":"26052ac7-68a8-45cf-b5f7-78d619964ec4","Type":"ContainerDied","Data":"f15a8bf028559df7cb3cfa021226b5805480b3c4463dcc125dc8b5ea3f44ef3e"}
Jul 28 12:21:31.780061 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.779960    2552 scope.go:117] "RemoveContainer" containerID="34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"
Jul 28 12:21:31.789177 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.789159    2552 scope.go:117] "RemoveContainer" containerID="34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"
Jul 28 12:21:31.789424 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:21:31.789407    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113\": container with ID starting with 34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113 not found: ID does not exist" containerID="34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"
Jul 28 12:21:31.789465 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.789434    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113"} err="failed to get container status \"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113\": rpc error: code = NotFound desc = could not find container \"34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113\": container with ID starting with 34ccdb610d95ad6f1acf4772a4643bbb3ee47cf47bf0b0a04e140b9b4307a113 not found: ID does not exist"
Jul 28 12:21:31.800850 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.800823    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:21:31.804417 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:31.804395    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/auth-test-model-e7af0c-kserve-54fbc7d99f-pd5pk"]
Jul 28 12:21:32.108836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:32.108754    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26052ac7-68a8-45cf-b5f7-78d619964ec4" path="/var/lib/kubelet/pods/26052ac7-68a8-45cf-b5f7-78d619964ec4/volumes"
Jul 28 12:21:38.705953 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:38.705909    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:38.706400 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:38.705969    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:38.719367 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:38.719327    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:21:38.823428 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:21:38.823396    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:23:09.162232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:09.162193    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:23:24.185696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.185657    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" podUID="78262aab-7b84-416f-b616-1e1edf64a805" containerName="main" containerID="cri-o://89760a688fe054650057a1506fff28edffeac4d3dd1afcbb68730dcdc9fa1b81" gracePeriod=45
Jul 28 12:23:24.316122 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.316079    2552 generic.go:356] "Generic (PLEG): container finished" podID="78262aab-7b84-416f-b616-1e1edf64a805" containerID="89760a688fe054650057a1506fff28edffeac4d3dd1afcbb68730dcdc9fa1b81" exitCode=0
Jul 28 12:23:24.316264 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.316159    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" event={"ID":"78262aab-7b84-416f-b616-1e1edf64a805","Type":"ContainerDied","Data":"89760a688fe054650057a1506fff28edffeac4d3dd1afcbb68730dcdc9fa1b81"}
Jul 28 12:23:24.437825 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.437753    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:23:24.507560 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507526    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhnc2\" (UniqueName: \"kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.507753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507575    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.507753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507638    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.507753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507677    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.507753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507698    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.507753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.507717    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache\") pod \"78262aab-7b84-416f-b616-1e1edf64a805\" (UID: \"78262aab-7b84-416f-b616-1e1edf64a805\") "
Jul 28 12:23:24.508064 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.508001    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:23:24.508064 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.508021    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home" (OuterVolumeSpecName: "home") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:23:24.508172 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.508101    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache" (OuterVolumeSpecName: "model-cache") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:23:24.509802 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.509770    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm" (OuterVolumeSpecName: "dshm") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:23:24.509946 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.509844    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:23:24.509946 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.509849    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2" (OuterVolumeSpecName: "kube-api-access-jhnc2") pod "78262aab-7b84-416f-b616-1e1edf64a805" (UID: "78262aab-7b84-416f-b616-1e1edf64a805"). InnerVolumeSpecName "kube-api-access-jhnc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:23:24.608821 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608778    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-tmp-dir\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:24.608821 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608813    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78262aab-7b84-416f-b616-1e1edf64a805-tls-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:24.608821 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608825    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-model-cache\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:24.609082 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608837    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhnc2\" (UniqueName: \"kubernetes.io/projected/78262aab-7b84-416f-b616-1e1edf64a805-kube-api-access-jhnc2\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:24.609082 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608849    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-dshm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:24.609082 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:24.608861    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78262aab-7b84-416f-b616-1e1edf64a805-home\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:23:25.323531 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:25.323492    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6" event={"ID":"78262aab-7b84-416f-b616-1e1edf64a805","Type":"ContainerDied","Data":"86d805653c9c28c92b8b56132cad2ffb96805ec11ab476aef6d47a5431190818"}
Jul 28 12:23:25.323531 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:25.323537    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"
Jul 28 12:23:25.324040 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:25.323557    2552 scope.go:117] "RemoveContainer" containerID="89760a688fe054650057a1506fff28edffeac4d3dd1afcbb68730dcdc9fa1b81"
Jul 28 12:23:25.344400 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:25.344371    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:23:25.348412 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:25.348390    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4d5cfe/sub-test-model-4d5cfe-kserve-6b6b7cd8b5-95vf6"]
Jul 28 12:23:26.108948 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:26.108881    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78262aab-7b84-416f-b616-1e1edf64a805" path="/var/lib/kubelet/pods/78262aab-7b84-416f-b616-1e1edf64a805/volumes"
Jul 28 12:23:42.515259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515226    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515598    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26052ac7-68a8-45cf-b5f7-78d619964ec4" containerName="main"
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515608    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="26052ac7-68a8-45cf-b5f7-78d619964ec4" containerName="main"
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515615    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78262aab-7b84-416f-b616-1e1edf64a805" containerName="main"
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515621    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="78262aab-7b84-416f-b616-1e1edf64a805" containerName="main"
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515680    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="26052ac7-68a8-45cf-b5f7-78d619964ec4" containerName="main"
Jul 28 12:23:42.515728 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.515689    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="78262aab-7b84-416f-b616-1e1edf64a805" containerName="main"
Jul 28 12:23:42.520035 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.520016    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.522950 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.522927    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"kube-root-ca.crt\""
Jul 28 12:23:42.522950 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.522938    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"default-dockercfg-sn6d8\""
Jul 28 12:23:42.523147 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.523035    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"rate-test-model-e7af0c-kserve-self-signed-certs\""
Jul 28 12:23:42.523147 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.523090    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e7af0c\"/\"openshift-service-ca.crt\""
Jul 28 12:23:42.528812 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.528791    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:23:42.568358 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568328    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.568515 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568369    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.568515 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568391    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.568515 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568461    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.568654 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568537    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.568654 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.568573    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzj6\" (UniqueName: \"kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.669705 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.669666    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.670046 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670022    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzj6\" (UniqueName: \"kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.670587 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670561    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.670710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.670710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670640    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.670710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670684    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.671236 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.670355    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.671484 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.671455    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.671655 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.671629    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.673580 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.673548    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.673692 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.673589    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.679586 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.679552    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzj6\" (UniqueName: \"kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6\") pod \"rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") " pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.829921 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.829824    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:42.963239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:42.963191    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:23:42.969212 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:23:42.969147    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf41e29ea_b24f_4977_878b_2c3d9821be63.slice/crio-9c83682d28093ce5daad48c1c8806eae76aa7d8667032e7b7840c9f159b85277 WatchSource:0}: Error finding container 9c83682d28093ce5daad48c1c8806eae76aa7d8667032e7b7840c9f159b85277: Status 404 returned error can't find the container with id 9c83682d28093ce5daad48c1c8806eae76aa7d8667032e7b7840c9f159b85277
Jul 28 12:23:43.408038 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:43.408004    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" event={"ID":"f41e29ea-b24f-4977-878b-2c3d9821be63","Type":"ContainerStarted","Data":"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"}
Jul 28 12:23:43.408038 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:43.408048    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" event={"ID":"f41e29ea-b24f-4977-878b-2c3d9821be63","Type":"ContainerStarted","Data":"9c83682d28093ce5daad48c1c8806eae76aa7d8667032e7b7840c9f159b85277"}
Jul 28 12:23:43.423532 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:43.423487    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" podStartSLOduration=1.42347058 podStartE2EDuration="1.42347058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:23:43.42283719 +0000 UTC m=+3187.879871647" watchObservedRunningTime="2026-07-28 12:23:43.42347058 +0000 UTC m=+3187.880505036"
Jul 28 12:23:52.830710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:52.830661    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:52.830710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:52.830723    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:52.843736 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:52.843710    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:23:53.466646 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:23:53.466616    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:24:54.796853 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:24:54.796771    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:25:09.815184 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:09.815138    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" podUID="f41e29ea-b24f-4977-878b-2c3d9821be63" containerName="main" containerID="cri-o://419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6" gracePeriod=45
Jul 28 12:25:10.057473 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.057449    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:25:10.149635 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149557    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmzj6\" (UniqueName: \"kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.149635 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149598    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.149635 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149628    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.149935 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149643    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.149935 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149688    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.149935 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149722    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home\") pod \"f41e29ea-b24f-4977-878b-2c3d9821be63\" (UID: \"f41e29ea-b24f-4977-878b-2c3d9821be63\") "
Jul 28 12:25:10.150116 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.149987    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:25:10.150116 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.150058    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home" (OuterVolumeSpecName: "home") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:25:10.150218 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.150132    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-tmp-dir\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.150218 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.150150    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-home\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.150326 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.150262    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache" (OuterVolumeSpecName: "model-cache") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:25:10.151833 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.151800    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6" (OuterVolumeSpecName: "kube-api-access-dmzj6") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "kube-api-access-dmzj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:25:10.151962 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.151841    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:25:10.151962 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.151883    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm" (OuterVolumeSpecName: "dshm") pod "f41e29ea-b24f-4977-878b-2c3d9821be63" (UID: "f41e29ea-b24f-4977-878b-2c3d9821be63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:25:10.250581 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.250548    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmzj6\" (UniqueName: \"kubernetes.io/projected/f41e29ea-b24f-4977-878b-2c3d9821be63-kube-api-access-dmzj6\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.250581 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.250576    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f41e29ea-b24f-4977-878b-2c3d9821be63-tls-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.250581 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.250587    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-model-cache\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.250801 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.250595    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f41e29ea-b24f-4977-878b-2c3d9821be63-dshm\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:25:10.814395 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.814357    2552 generic.go:356] "Generic (PLEG): container finished" podID="f41e29ea-b24f-4977-878b-2c3d9821be63" containerID="419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6" exitCode=0
Jul 28 12:25:10.814588 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.814403    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"
Jul 28 12:25:10.814588 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.814453    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" event={"ID":"f41e29ea-b24f-4977-878b-2c3d9821be63","Type":"ContainerDied","Data":"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"}
Jul 28 12:25:10.814588 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.814497    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z" event={"ID":"f41e29ea-b24f-4977-878b-2c3d9821be63","Type":"ContainerDied","Data":"9c83682d28093ce5daad48c1c8806eae76aa7d8667032e7b7840c9f159b85277"}
Jul 28 12:25:10.814588 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.814514    2552 scope.go:117] "RemoveContainer" containerID="419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"
Jul 28 12:25:10.824023 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.823852    2552 scope.go:117] "RemoveContainer" containerID="419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"
Jul 28 12:25:10.824296 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:25:10.824143    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6\": container with ID starting with 419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6 not found: ID does not exist" containerID="419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"
Jul 28 12:25:10.824296 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.824181    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6"} err="failed to get container status \"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6\": rpc error: code = NotFound desc = could not find container \"419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6\": container with ID starting with 419e79f93d1920b07251fd796ca8db9d8df8030eea4b467db32e5c683e91daa6 not found: ID does not exist"
Jul 28 12:25:10.833996 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.833972    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:25:10.837778 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:10.837756    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e7af0c/rate-test-model-e7af0c-kserve-7b5bbff8dc-hss5z"]
Jul 28 12:25:12.108904 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:12.108868    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f41e29ea-b24f-4977-878b-2c3d9821be63" path="/var/lib/kubelet/pods/f41e29ea-b24f-4977-878b-2c3d9821be63/volumes"
Jul 28 12:25:14.660140 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.660103    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:25:14.660522 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.660502    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f41e29ea-b24f-4977-878b-2c3d9821be63" containerName="main"
Jul 28 12:25:14.660522 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.660514    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f41e29ea-b24f-4977-878b-2c3d9821be63" containerName="main"
Jul 28 12:25:14.660596 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.660578    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f41e29ea-b24f-4977-878b-2c3d9821be63" containerName="main"
Jul 28 12:25:14.664906 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.664880    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.668239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.668220    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-ba0125-openshift-default-dockercfg-cqn6s\""
Jul 28 12:25:14.692075 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692054    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692179 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692087    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692179 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692109    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692179 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692128    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692204    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692255    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692287 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692270    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692381 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692321    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.692381 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.692342    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvk9v\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.697742 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.697718    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:25:14.793615 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793626    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793682    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvk9v\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793734    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793776    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.793837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.793826    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794004    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794058    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794232 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794407 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794318    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794407 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794384    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794511 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794429    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.794671 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.794654    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.796554 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.796533    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.796859 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.796837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.801397 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.801369    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.801836 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.801808    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvk9v\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v\") pod \"e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") " pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:14.975403 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:14.975363    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:15.111600 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.111541    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:25:15.113943 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:25:15.113909    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4cc2c4_d888_4d84_8e09_04948b352de2.slice/crio-74f53bcc5b395dba7b6362b3e0c9a50b952c0788d318e32683eb8da413370791 WatchSource:0}: Error finding container 74f53bcc5b395dba7b6362b3e0c9a50b952c0788d318e32683eb8da413370791: Status 404 returned error can't find the container with id 74f53bcc5b395dba7b6362b3e0c9a50b952c0788d318e32683eb8da413370791
Jul 28 12:25:15.116388 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.116344    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:15.116470 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.116434    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:15.116516 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.116475    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:15.842391 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.842347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerStarted","Data":"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"}
Jul 28 12:25:15.842770 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.842417    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerStarted","Data":"74f53bcc5b395dba7b6362b3e0c9a50b952c0788d318e32683eb8da413370791"}
Jul 28 12:25:15.863751 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.863696    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podStartSLOduration=1.86368161 podStartE2EDuration="1.86368161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:25:15.861193071 +0000 UTC m=+3280.318227541" watchObservedRunningTime="2026-07-28 12:25:15.86368161 +0000 UTC m=+3280.320716066"
Jul 28 12:25:15.975579 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:15.975531    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:16.981543 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:16.981511    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:17.852445 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:17.852409    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:17.853450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:17.853425    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:33.853050 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:33.853003    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 12:25:33.853538 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:33.853096    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 12:25:34.935678 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.935643    2552 generic.go:356] "Generic (PLEG): container finished" podID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerID="675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc" exitCode=0
Jul 28 12:25:34.936123 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.935718    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerDied","Data":"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"}
Jul 28 12:25:34.936186 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.936148    2552 scope.go:117] "RemoveContainer" containerID="675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"
Jul 28 12:25:34.937295 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.937269    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:34.937355 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.937336    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:34.937395 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:34.937366    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:25:35.947606 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:35.947552    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerStarted","Data":"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"}
Jul 28 12:25:35.975814 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:35.975775    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:36.976314 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:36.976277    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 28 12:25:36.976723 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:36.976358    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 28 12:25:37.976228 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:37.976198    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 28 12:25:37.976843 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:37.976815    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 28 12:25:38.976115 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:38.976071    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 28 12:25:38.976681 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:38.976631    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 28 12:25:39.976621 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:39.976584    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 28 12:25:39.977087 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:39.976655    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 28 12:25:40.979994 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:40.979962    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:40.980478 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:40.980230    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:25:40.980813 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:25:40.980798    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:26:50.021600 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.021564    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:26:50.025059 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.025041    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.027403 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.027379    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-0f3ba4-openshift-default-dockercfg-9wzkc\""
Jul 28 12:26:50.048627 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.048596    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:26:50.078503 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078477    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078647 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078508    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078647 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078526    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078647 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078590    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078768 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078712    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078808 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078761    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078844 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078812    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f8np\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078879 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078844    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.078990 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.078975    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180181 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180144    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180206    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f8np\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180238    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180355    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180385    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180429    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180487    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180581    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180580    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180669 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180628    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.180954 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.180855    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.181192 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.181166    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.181250 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.181222    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.182819 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.182796    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.183137 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.183119    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.190660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.190636    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f8np\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.194513 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.194489    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token\") pod \"e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") " pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.336022 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.335920    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:50.471969 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.471940    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:26:50.474005 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:26:50.473976    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78035b6f_c008_47b2_88fe_8c8897d474c3.slice/crio-76cd5d87e9e368a4a8e02b83504ba070a86dd3c5a1285bae1afdcca631354f67 WatchSource:0}: Error finding container 76cd5d87e9e368a4a8e02b83504ba070a86dd3c5a1285bae1afdcca631354f67: Status 404 returned error can't find the container with id 76cd5d87e9e368a4a8e02b83504ba070a86dd3c5a1285bae1afdcca631354f67
Jul 28 12:26:50.476256 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.476223    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:26:50.476334 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.476292    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:26:50.476334 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:50.476321    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:26:51.316460 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:51.316415    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerStarted","Data":"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"}
Jul 28 12:26:51.316857 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:51.316480    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerStarted","Data":"76cd5d87e9e368a4a8e02b83504ba070a86dd3c5a1285bae1afdcca631354f67"}
Jul 28 12:26:51.336620 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:51.336567    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podStartSLOduration=2.336552012 podStartE2EDuration="2.336552012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:26:51.33422298 +0000 UTC m=+3375.791257437" watchObservedRunningTime="2026-07-28 12:26:51.336552012 +0000 UTC m=+3375.793586469"
Jul 28 12:26:51.337043 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:51.337023    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:51.342139 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:51.342116    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:52.321392 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:52.321358    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:26:52.322539 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:26:52.322520    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:07.323179 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.323136    2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 28 12:27:07.323633 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.323198    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 28 12:27:07.398597 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.398562    2552 generic.go:356] "Generic (PLEG): container finished" podID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerID="e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642" exitCode=0
Jul 28 12:27:07.398761 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.398639    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerDied","Data":"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"}
Jul 28 12:27:07.399063 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.399048    2552 scope.go:117] "RemoveContainer" containerID="e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"
Jul 28 12:27:07.400121 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.400090    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:27:07.400233 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.400173    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:27:07.400233 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:07.400224    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 12:27:08.405563 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:08.405528    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerStarted","Data":"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"}
Jul 28 12:27:09.336914 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:09.336864    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:09.337263 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:09.337239    2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 28 12:27:09.337442 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:09.337417    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 28 12:27:10.337081 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:10.337037    2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 28 12:27:10.337560 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:10.337110    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 28 12:27:11.336306 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:11.336267    2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 28 12:27:11.336603 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:11.336571    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 28 12:27:12.336371 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:12.336327    2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 28 12:27:12.336912 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:12.336400    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 28 12:27:13.340872 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:13.340834    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:13.341259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:13.341097    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:13.341848 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:13.341828    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:31.208607 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:31.208527    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:27:31.209005 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:31.208792    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy" containerID="cri-o://1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3" gracePeriod=30
Jul 28 12:27:36.474257 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.474188    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:36.542156 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.542119    2552 generic.go:356] "Generic (PLEG): container finished" podID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerID="1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3" exitCode=0
Jul 28 12:27:36.542320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.542171    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"
Jul 28 12:27:36.542320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.542176    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerDied","Data":"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"}
Jul 28 12:27:36.542320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.542225    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77" event={"ID":"78035b6f-c008-47b2-88fe-8c8897d474c3","Type":"ContainerDied","Data":"76cd5d87e9e368a4a8e02b83504ba070a86dd3c5a1285bae1afdcca631354f67"}
Jul 28 12:27:36.542320 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.542248    2552 scope.go:117] "RemoveContainer" containerID="1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"
Jul 28 12:27:36.551313 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.551299    2552 scope.go:117] "RemoveContainer" containerID="e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"
Jul 28 12:27:36.559748 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.559724    2552 scope.go:117] "RemoveContainer" containerID="1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"
Jul 28 12:27:36.560016 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:27:36.559996    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3\": container with ID starting with 1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3 not found: ID does not exist" containerID="1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"
Jul 28 12:27:36.560086 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.560027    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3"} err="failed to get container status \"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3\": rpc error: code = NotFound desc = could not find container \"1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3\": container with ID starting with 1cd4a9effbb858f4c2efdc5f842336256275484316e582d06c5f1652433563c3 not found: ID does not exist"
Jul 28 12:27:36.560086 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.560046    2552 scope.go:117] "RemoveContainer" containerID="e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"
Jul 28 12:27:36.560309 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:27:36.560290    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642\": container with ID starting with e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642 not found: ID does not exist" containerID="e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"
Jul 28 12:27:36.560352 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.560314    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642"} err="failed to get container status \"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642\": rpc error: code = NotFound desc = could not find container \"e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642\": container with ID starting with e3d7e4bcf09470f3ee0cf97f65dd33044c2ef1478f2e5061f9a1e2994c4b1642 not found: ID does not exist"
Jul 28 12:27:36.629281 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629248    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629462 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629297    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629462 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629318    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f8np\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629462 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629349    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629462 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629397    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629462 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629415    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629473    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629502    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629558    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data\") pod \"78035b6f-c008-47b2-88fe-8c8897d474c3\" (UID: \"78035b6f-c008-47b2-88fe-8c8897d474c3\") "
Jul 28 12:27:36.629740 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629574    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:27:36.630017 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629738    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:27:36.630017 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629922    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.630017 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.629990    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.630180 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.630034    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:27:36.630293 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.630243    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:27:36.630637 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.630610    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data" (OuterVolumeSpecName: "istio-data") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:27:36.631650 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.631624    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:27:36.631753 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.631668    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:27:36.631806 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.631750    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:27:36.632051 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.632030    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np" (OuterVolumeSpecName: "kube-api-access-2f8np") pod "78035b6f-c008-47b2-88fe-8c8897d474c3" (UID: "78035b6f-c008-47b2-88fe-8c8897d474c3"). InnerVolumeSpecName "kube-api-access-2f8np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731421    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731444    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731453    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731463    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731473    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f8np\" (UniqueName: \"kubernetes.io/projected/78035b6f-c008-47b2-88fe-8c8897d474c3-kube-api-access-2f8np\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731485    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78035b6f-c008-47b2-88fe-8c8897d474c3-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.731490 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.731494    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78035b6f-c008-47b2-88fe-8c8897d474c3-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:27:36.866027 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.865986    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:27:36.872048 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:36.872014    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0f3ba4-openshift-default-5c74d598dd-vwg77"]
Jul 28 12:27:38.108814 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:27:38.108779    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" path="/var/lib/kubelet/pods/78035b6f-c008-47b2-88fe-8c8897d474c3/volumes"
Jul 28 12:28:45.551379 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:45.551343    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:28:45.551959 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:45.551582    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy" containerID="cri-o://7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42" gracePeriod=30
Jul 28 12:28:50.818859 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.818795    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:28:50.893421 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.893384    2552 generic.go:356] "Generic (PLEG): container finished" podID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerID="7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42" exitCode=0
Jul 28 12:28:50.893700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.893444    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"
Jul 28 12:28:50.893700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.893456    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerDied","Data":"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"}
Jul 28 12:28:50.893700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.893542    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6" event={"ID":"1b4cc2c4-d888-4d84-8e09-04948b352de2","Type":"ContainerDied","Data":"74f53bcc5b395dba7b6362b3e0c9a50b952c0788d318e32683eb8da413370791"}
Jul 28 12:28:50.893700 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.893577    2552 scope.go:117] "RemoveContainer" containerID="7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"
Jul 28 12:28:50.902013 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.901992    2552 scope.go:117] "RemoveContainer" containerID="675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"
Jul 28 12:28:50.904133 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904111    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904227 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904146    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904227 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904209    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904227 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904225    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904242    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904280    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvk9v\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904323    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904394 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904346    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904407    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token\") pod \"1b4cc2c4-d888-4d84-8e09-04948b352de2\" (UID: \"1b4cc2c4-d888-4d84-8e09-04948b352de2\") "
Jul 28 12:28:50.904610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904520    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:28:50.904610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904585    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:28:50.904610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904600    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:28:50.904826 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904798    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:50.904826 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904819    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4cc2c4-d888-4d84-8e09-04948b352de2-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:50.904952 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904827    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:28:50.904952 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.904835    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:50.905536 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.905502    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:28:50.906422 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.906397    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:28:50.906696 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.906670    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:28:50.906994 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.906976    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:28:50.907047 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.907030    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v" (OuterVolumeSpecName: "kube-api-access-rvk9v") pod "1b4cc2c4-d888-4d84-8e09-04948b352de2" (UID: "1b4cc2c4-d888-4d84-8e09-04948b352de2"). InnerVolumeSpecName "kube-api-access-rvk9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:28:50.924224 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.924205    2552 scope.go:117] "RemoveContainer" containerID="7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"
Jul 28 12:28:50.924479 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:28:50.924458    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42\": container with ID starting with 7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42 not found: ID does not exist" containerID="7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"
Jul 28 12:28:50.924562 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.924490    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42"} err="failed to get container status \"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42\": rpc error: code = NotFound desc = could not find container \"7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42\": container with ID starting with 7d8af67ac57513c86351a646209dcf5316efdbfebeca0c7f0197a4f603a39d42 not found: ID does not exist"
Jul 28 12:28:50.924562 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.924516    2552 scope.go:117] "RemoveContainer" containerID="675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"
Jul 28 12:28:50.924796 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:28:50.924778    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc\": container with ID starting with 675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc not found: ID does not exist" containerID="675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"
Jul 28 12:28:50.924839 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:50.924802    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc"} err="failed to get container status \"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc\": rpc error: code = NotFound desc = could not find container \"675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc\": container with ID starting with 675c1765d8480e5b0cfbf457c8aea0f7a3e9564d024091c6caaebe5ad61d89dc not found: ID does not exist"
Jul 28 12:28:51.005262 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005234    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.005262 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005258    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.005262 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005268    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.005450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005276    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.005450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005283    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4cc2c4-d888-4d84-8e09-04948b352de2-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.005450 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.005292    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvk9v\" (UniqueName: \"kubernetes.io/projected/1b4cc2c4-d888-4d84-8e09-04948b352de2-kube-api-access-rvk9v\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:28:51.222342 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.222305    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:28:51.226493 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:51.226468    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-ba0125-openshift-default-76c49879cb-7drw6"]
Jul 28 12:28:52.108665 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:28:52.108631    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" path="/var/lib/kubelet/pods/1b4cc2c4-d888-4d84-8e09-04948b352de2/volumes"
Jul 28 12:29:30.825371 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:30.825338    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:29:30.825868 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:30.825549    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" containerID="cri-o://5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e" gracePeriod=30
Jul 28 12:29:31.503820 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:31.503779    2552 patch_prober.go:28] interesting pod/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 12:29:31.504010 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:31.503843    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 12:29:36.078147 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.078123    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:29:36.114437 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.114336    2552 generic.go:356] "Generic (PLEG): container finished" podID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerID="5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e" exitCode=0
Jul 28 12:29:36.114624 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.114444    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"
Jul 28 12:29:36.118581 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.118552    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerDied","Data":"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"}
Jul 28 12:29:36.118704 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.118612    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j" event={"ID":"e1d317fd-174f-4422-945c-7fb821a9d08e","Type":"ContainerDied","Data":"b3be519140cd11be753d2aea611df1be9cf90d65d05a74edb7899f5c2a15af5d"}
Jul 28 12:29:36.118704 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.118648    2552 scope.go:117] "RemoveContainer" containerID="5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"
Jul 28 12:29:36.127213 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.127196    2552 scope.go:117] "RemoveContainer" containerID="0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"
Jul 28 12:29:36.134675 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.134661    2552 scope.go:117] "RemoveContainer" containerID="5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"
Jul 28 12:29:36.134965 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:29:36.134941    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e\": container with ID starting with 5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e not found: ID does not exist" containerID="5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"
Jul 28 12:29:36.135029 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.134979    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e"} err="failed to get container status \"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e\": rpc error: code = NotFound desc = could not find container \"5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e\": container with ID starting with 5d4470caf82b1dc795a901400b102080cb2a1d11fbaacc8829381515e05fea5e not found: ID does not exist"
Jul 28 12:29:36.135029 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.135004    2552 scope.go:117] "RemoveContainer" containerID="0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"
Jul 28 12:29:36.135255 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:29:36.135227    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd\": container with ID starting with 0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd not found: ID does not exist" containerID="0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"
Jul 28 12:29:36.135297 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.135265    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd"} err="failed to get container status \"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd\": rpc error: code = NotFound desc = could not find container \"0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd\": container with ID starting with 0e70cd7366c35c4ebe319121ae391881cebc124f5bd238525d37747222fd9bbd not found: ID does not exist"
Jul 28 12:29:36.203129 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203101    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203278 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203144    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203278 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203167    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203278 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203203    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203448 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203298    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203448 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203325    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203448 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203355    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203448 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203404    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9r2q\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203461    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy\") pod \"e1d317fd-174f-4422-945c-7fb821a9d08e\" (UID: \"e1d317fd-174f-4422-945c-7fb821a9d08e\") "
Jul 28 12:29:36.203660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203481    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:36.203660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203605    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:36.203660 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203622    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:29:36.203884 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203732    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:36.203884 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203842    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1d317fd-174f-4422-945c-7fb821a9d08e-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.203884 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203862    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.203884 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203874    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.204114 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.203887    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.204259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.204236    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:36.205465 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.205447    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q" (OuterVolumeSpecName: "kube-api-access-f9r2q") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "kube-api-access-f9r2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:29:36.205574 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.205554    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:29:36.205713 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.205691    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:29:36.205821 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.205769    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1d317fd-174f-4422-945c-7fb821a9d08e" (UID: "e1d317fd-174f-4422-945c-7fb821a9d08e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:36.305239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.305204    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.305239 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.305233    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9r2q\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-kube-api-access-f9r2q\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.305426 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.305246    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.305426 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.305259    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.305426 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.305270    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1d317fd-174f-4422-945c-7fb821a9d08e-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:36.437799 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.437775    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:29:36.442162 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:36.442139    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4d5cfe-openshift-default-6bd7fdbf6d-kmz6j"]
Jul 28 12:29:38.109206 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:38.109165    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" path="/var/lib/kubelet/pods/e1d317fd-174f-4422-945c-7fb821a9d08e/volumes"
Jul 28 12:29:53.300076 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:53.300038    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:29:53.300483 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:53.300262    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy" containerID="cri-o://224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c" gracePeriod=30
Jul 28 12:29:58.546280 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.546255    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:29:58.703586 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703546    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703586 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703596    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703631    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703660    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703693    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703767    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703785    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqt22\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.703837 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703815    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.704199 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.703844    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs\") pod \"215971c1-dd5b-4944-ad59-9220a7bb8974\" (UID: \"215971c1-dd5b-4944-ad59-9220a7bb8974\") "
Jul 28 12:29:58.704199 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704001    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:58.704199 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704094    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:58.704372 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704207    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:29:58.704372 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704224    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:58.704372 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704224    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.704372 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704260    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-credential-socket\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.704779 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.704759    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data" (OuterVolumeSpecName: "istio-data") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:58.705879 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.705852    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 12:29:58.705977 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.705883    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:29:58.706111 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.706091    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token" (OuterVolumeSpecName: "istio-token") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:29:58.706346 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.706328    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22" (OuterVolumeSpecName: "kube-api-access-nqt22") pod "215971c1-dd5b-4944-ad59-9220a7bb8974" (UID: "215971c1-dd5b-4944-ad59-9220a7bb8974"). InnerVolumeSpecName "kube-api-access-nqt22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:29:58.805611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805581    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqt22\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-kube-api-access-nqt22\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805611 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805612    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-data\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805795 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805622    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-workload-certs\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805795 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805630    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-token\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805795 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805638    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-podinfo\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805795 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805645    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/215971c1-dd5b-4944-ad59-9220a7bb8974-istio-envoy\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:58.805795 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:58.805653    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/215971c1-dd5b-4944-ad59-9220a7bb8974-istiod-ca-cert\") on node \"ip-10-0-130-140.ec2.internal\" DevicePath \"\""
Jul 28 12:29:59.220132 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.220099    2552 generic.go:356] "Generic (PLEG): container finished" podID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerID="224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c" exitCode=0
Jul 28 12:29:59.220306 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.220143    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"
Jul 28 12:29:59.220306 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.220173    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerDied","Data":"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"}
Jul 28 12:29:59.220306 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.220224    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz" event={"ID":"215971c1-dd5b-4944-ad59-9220a7bb8974","Type":"ContainerDied","Data":"e8683fb7bd4c66e2fc82e80b55a6d1b242871387c32b1239cda9cdf7b150fea2"}
Jul 28 12:29:59.220306 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.220247    2552 scope.go:117] "RemoveContainer" containerID="224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"
Jul 28 12:29:59.229139 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.229120    2552 scope.go:117] "RemoveContainer" containerID="6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"
Jul 28 12:29:59.236938 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.236921    2552 scope.go:117] "RemoveContainer" containerID="224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"
Jul 28 12:29:59.237159 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:29:59.237143    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c\": container with ID starting with 224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c not found: ID does not exist" containerID="224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"
Jul 28 12:29:59.237217 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.237167    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c"} err="failed to get container status \"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c\": rpc error: code = NotFound desc = could not find container \"224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c\": container with ID starting with 224aeecc13267e62f3af0f41a32e36d57b791b9ec02161ea6caf5f307140723c not found: ID does not exist"
Jul 28 12:29:59.237217 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.237189    2552 scope.go:117] "RemoveContainer" containerID="6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"
Jul 28 12:29:59.237398 ip-10-0-130-140 kubenswrapper[2552]: E0728 12:29:59.237382    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87\": container with ID starting with 6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87 not found: ID does not exist" containerID="6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"
Jul 28 12:29:59.237444 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.237402    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87"} err="failed to get container status \"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87\": rpc error: code = NotFound desc = could not find container \"6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87\": container with ID starting with 6f175309f871960de26958cdb08b8ded845b219d206023bbb7fc592114a10a87 not found: ID does not exist"
Jul 28 12:29:59.240889 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.240867    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:29:59.245186 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:29:59.245165    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e7af0c-openshift-default-8655db77b-7kksz"]
Jul 28 12:30:00.108856 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:00.108826    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" path="/var/lib/kubelet/pods/215971c1-dd5b-4944-ad59-9220a7bb8974/volumes"
Jul 28 12:30:00.240642 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:00.240612    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6954b9867f-tzmg8_c24e3e82-e29a-4a5a-b3f4-d09cb354139f/manager/0.log"
Jul 28 12:30:00.873417 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:00.873387    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6755d7577-jfn6z_624b4bc4-e9f3-447b-b96d-2a5e35dc4d60/copy-manifests/0.log"
Jul 28 12:30:00.886756 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:00.886728    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6755d7577-jfn6z_624b4bc4-e9f3-447b-b96d-2a5e35dc4d60/manager/0.log"
Jul 28 12:30:04.636718 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:04.636682    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-v48hg_6f95e161-5c34-4fce-915f-c37e57a514c9/discovery/0.log"
Jul 28 12:30:04.883349 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:04.883322    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8489dc4db9-rbztv_93386518-d4ed-4ceb-8706-e8daf0821ff9/kube-auth-proxy/0.log"
Jul 28 12:30:12.475343 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475307    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwgdc/must-gather-x5n78"]
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475649    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475660    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475669    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475675    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475682    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475689    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475705    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475711    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475719    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.475727 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475724    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475735    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475740    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475798    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475804    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475811    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475816    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475822    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="78035b6f-c008-47b2-88fe-8c8897d474c3" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475829    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475890    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475910    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1d317fd-174f-4422-945c-7fb821a9d08e" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475919    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475927    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475990    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="215971c1-dd5b-4944-ad59-9220a7bb8974" containerName="istio-proxy"
Jul 28 12:30:12.476068 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.475995    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4cc2c4-d888-4d84-8e09-04948b352de2" containerName="istio-proxy"
Jul 28 12:30:12.480402 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.480379    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.483464 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.483443    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dwgdc\"/\"default-dockercfg-6rr66\""
Jul 28 12:30:12.483963 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.483947    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwgdc\"/\"kube-root-ca.crt\""
Jul 28 12:30:12.484057 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.483987    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwgdc\"/\"openshift-service-ca.crt\""
Jul 28 12:30:12.494656 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.494613    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwgdc/must-gather-x5n78"]
Jul 28 12:30:12.516481 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.516440    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69hc6\" (UniqueName: \"kubernetes.io/projected/94e6497f-d143-4146-ae28-d1871d7007b7-kube-api-access-69hc6\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.516651 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.516523    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/94e6497f-d143-4146-ae28-d1871d7007b7-must-gather-output\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.617418 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.617378    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69hc6\" (UniqueName: \"kubernetes.io/projected/94e6497f-d143-4146-ae28-d1871d7007b7-kube-api-access-69hc6\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.617613 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.617457    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/94e6497f-d143-4146-ae28-d1871d7007b7-must-gather-output\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.617739 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.617721    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/94e6497f-d143-4146-ae28-d1871d7007b7-must-gather-output\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.625710 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.625675    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69hc6\" (UniqueName: \"kubernetes.io/projected/94e6497f-d143-4146-ae28-d1871d7007b7-kube-api-access-69hc6\") pod \"must-gather-x5n78\" (UID: \"94e6497f-d143-4146-ae28-d1871d7007b7\") " pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.790042 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.789956    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwgdc/must-gather-x5n78"
Jul 28 12:30:12.926334 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.926303    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwgdc/must-gather-x5n78"]
Jul 28 12:30:12.928379 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:30:12.928340    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94e6497f_d143_4146_ae28_d1871d7007b7.slice/crio-2ab3f4fd5d7ecbed6607eaa710fe8168adab132bfbaab3c076d4d648a9149672 WatchSource:0}: Error finding container 2ab3f4fd5d7ecbed6607eaa710fe8168adab132bfbaab3c076d4d648a9149672: Status 404 returned error can't find the container with id 2ab3f4fd5d7ecbed6607eaa710fe8168adab132bfbaab3c076d4d648a9149672
Jul 28 12:30:12.930208 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:12.930191    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:30:13.293610 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:13.293569    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwgdc/must-gather-x5n78" event={"ID":"94e6497f-d143-4146-ae28-d1871d7007b7","Type":"ContainerStarted","Data":"2ab3f4fd5d7ecbed6607eaa710fe8168adab132bfbaab3c076d4d648a9149672"}
Jul 28 12:30:14.300471 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:14.300431    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwgdc/must-gather-x5n78" event={"ID":"94e6497f-d143-4146-ae28-d1871d7007b7","Type":"ContainerStarted","Data":"10d7510145827dc5fb4017825e822e4107f6f58128e0194d3bf42c6dd7082af9"}
Jul 28 12:30:14.300979 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:14.300480    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwgdc/must-gather-x5n78" event={"ID":"94e6497f-d143-4146-ae28-d1871d7007b7","Type":"ContainerStarted","Data":"3ff62111191bd697d04d01e88c3b441967927f21ba9aa65df83d6d1941a70972"}
Jul 28 12:30:14.317544 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:14.317498    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwgdc/must-gather-x5n78" podStartSLOduration=1.488486521 podStartE2EDuration="2.317483194s" totalImagesPullingTime="828.996673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:14.315826492 +0000 UTC m=+3578.772860951" watchObservedRunningTime="2026-07-28 12:30:14.317483194 +0000 UTC m=+3578.774517650"
Jul 28 12:30:15.498745 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:15.498700    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rsxxg_2d198ec9-287f-46fe-8097-bdb6554c3c6c/global-pull-secret-syncer/0.log"
Jul 28 12:30:15.592678 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:15.592638    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-st4fs_2ac0c76b-c6f7-44b9-871c-86221abe73a8/konnectivity-agent/0.log"
Jul 28 12:30:15.660148 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:15.660119    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-140.ec2.internal_a16892ee32f85011fc1a1806c8666fbf/haproxy/0.log"
Jul 28 12:30:22.005926 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.005852    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/alertmanager/0.log"
Jul 28 12:30:22.030926 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.030867    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/config-reloader/0.log"
Jul 28 12:30:22.056119 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.056088    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/kube-rbac-proxy-web/0.log"
Jul 28 12:30:22.080363 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.080337    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/kube-rbac-proxy/0.log"
Jul 28 12:30:22.110797 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.110770    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/kube-rbac-proxy-metric/0.log"
Jul 28 12:30:22.137838 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.137808    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/prom-label-proxy/0.log"
Jul 28 12:30:22.172575 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.172544    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2428693a-1121-4121-acd8-6ff5714b0566/init-config-reloader/0.log"
Jul 28 12:30:22.364357 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.364255    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bd9489dd7-mpjxc_3b77eabb-3a87-4281-a736-5ab554dd3b92/metrics-server/0.log"
Jul 28 12:30:22.418995 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.418957    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-jsk9x_0852eab5-b13b-4ec4-9b26-f7aaae7ca8dc/monitoring-plugin/0.log"
Jul 28 12:30:22.588927 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.588882    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh496_a6feb4df-94c1-429a-88be-0c18fb25bc08/node-exporter/0.log"
Jul 28 12:30:22.616177 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.616084    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh496_a6feb4df-94c1-429a-88be-0c18fb25bc08/kube-rbac-proxy/0.log"
Jul 28 12:30:22.642514 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:22.642482    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qh496_a6feb4df-94c1-429a-88be-0c18fb25bc08/init-textfile/0.log"
Jul 28 12:30:23.124735 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.124702    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-7kls9_22915a2d-84a6-409e-89b1-57b48e1bb62e/prometheus-operator-admission-webhook/0.log"
Jul 28 12:30:23.162228 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.162199    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd9bfb7d8-kv8mr_da3246e1-4a3f-4d85-a60a-d6d210f29068/telemeter-client/0.log"
Jul 28 12:30:23.188484 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.188459    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd9bfb7d8-kv8mr_da3246e1-4a3f-4d85-a60a-d6d210f29068/reload/0.log"
Jul 28 12:30:23.221770 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.221743    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6dd9bfb7d8-kv8mr_da3246e1-4a3f-4d85-a60a-d6d210f29068/kube-rbac-proxy/0.log"
Jul 28 12:30:23.262012 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.261978    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/thanos-query/0.log"
Jul 28 12:30:23.288027 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.287977    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/kube-rbac-proxy-web/0.log"
Jul 28 12:30:23.318065 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.318035    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/kube-rbac-proxy/0.log"
Jul 28 12:30:23.356447 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.356414    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/prom-label-proxy/0.log"
Jul 28 12:30:23.387316 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.387239    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/kube-rbac-proxy-rules/0.log"
Jul 28 12:30:23.413240 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:23.413213    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f4bb6cd74-ks5l5_70ad7485-4bc8-4086-82ea-16b2e98157b7/kube-rbac-proxy-metrics/0.log"
Jul 28 12:30:24.692982 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:24.692939    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-9rfrd_e625692f-e4a7-46c7-b432-ef63006556ed/networking-console-plugin/0.log"
Jul 28 12:30:25.471217 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.471168    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"]
Jul 28 12:30:25.482054 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.482021    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"]
Jul 28 12:30:25.482420 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.482351    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.539114 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.539077    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-proc\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.539114 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.539122    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-podres\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.539332 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.539178    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-lib-modules\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.539332 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.539283    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhqtb\" (UniqueName: \"kubernetes.io/projected/c0384dd9-d850-4b8b-a485-f9290529d4a9-kube-api-access-lhqtb\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.539332 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.539318    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-sys\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640485 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640433    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhqtb\" (UniqueName: \"kubernetes.io/projected/c0384dd9-d850-4b8b-a485-f9290529d4a9-kube-api-access-lhqtb\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640485 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640486    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-sys\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640538    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-proc\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640568    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-podres\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-lib-modules\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640664    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-proc\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640744 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640663    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-sys\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640972 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640756    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-podres\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.640972 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.640762    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0384dd9-d850-4b8b-a485-f9290529d4a9-lib-modules\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.651478 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.651435    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhqtb\" (UniqueName: \"kubernetes.io/projected/c0384dd9-d850-4b8b-a485-f9290529d4a9-kube-api-access-lhqtb\") pod \"perf-node-gather-daemonset-mn9tl\" (UID: \"c0384dd9-d850-4b8b-a485-f9290529d4a9\") " pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.802632 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.802533    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:25.997259 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:25.997230    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"]
Jul 28 12:30:25.999438 ip-10-0-130-140 kubenswrapper[2552]: W0728 12:30:25.999405    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc0384dd9_d850_4b8b_a485_f9290529d4a9.slice/crio-873854d98ac1dd859378b5445174677d32238c93b26f6b9c4575f2767c7ea40b WatchSource:0}: Error finding container 873854d98ac1dd859378b5445174677d32238c93b26f6b9c4575f2767c7ea40b: Status 404 returned error can't find the container with id 873854d98ac1dd859378b5445174677d32238c93b26f6b9c4575f2767c7ea40b
Jul 28 12:30:26.386492 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:26.386378    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl" event={"ID":"c0384dd9-d850-4b8b-a485-f9290529d4a9","Type":"ContainerStarted","Data":"5f7c8b2bee88d1229a2841cb2bcfccaec4581ebaa2a96e504b8ded4656469282"}
Jul 28 12:30:26.386492 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:26.386472    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl" event={"ID":"c0384dd9-d850-4b8b-a485-f9290529d4a9","Type":"ContainerStarted","Data":"873854d98ac1dd859378b5445174677d32238c93b26f6b9c4575f2767c7ea40b"}
Jul 28 12:30:26.387476 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:26.387448    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:26.410451 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:26.410396    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl" podStartSLOduration=1.410380405 podStartE2EDuration="1.410380405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:26.408482396 +0000 UTC m=+3590.865516853" watchObservedRunningTime="2026-07-28 12:30:26.410380405 +0000 UTC m=+3590.867414904"
Jul 28 12:30:27.785847 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:27.785799    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9jzkc_f70ee650-e643-4722-bf6d-cfac90f5859c/dns/0.log"
Jul 28 12:30:27.809007 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:27.808973    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9jzkc_f70ee650-e643-4722-bf6d-cfac90f5859c/kube-rbac-proxy/0.log"
Jul 28 12:30:27.883054 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:27.883027    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-74fz4_06c990c3-78d5-4d21-93c9-a474055faa7a/dns-node-resolver/0.log"
Jul 28 12:30:28.559842 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:28.559751    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r4qzt_f6ed79d9-a93c-4e58-9cf8-b270064850fd/node-ca/0.log"
Jul 28 12:30:29.714504 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:29.714469    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-v48hg_6f95e161-5c34-4fce-915f-c37e57a514c9/discovery/0.log"
Jul 28 12:30:29.763102 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:29.763067    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8489dc4db9-rbztv_93386518-d4ed-4ceb-8706-e8daf0821ff9/kube-auth-proxy/0.log"
Jul 28 12:30:30.764916 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:30.764877    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rpzw5_8a63620b-e497-402b-9ea0-864e8f1df157/serve-healthcheck-canary/0.log"
Jul 28 12:30:31.485686 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:31.485645    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptmkj_771ef471-d88c-44c4-9506-99ff4adc16cf/kube-rbac-proxy/0.log"
Jul 28 12:30:31.514127 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:31.514093    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptmkj_771ef471-d88c-44c4-9506-99ff4adc16cf/exporter/0.log"
Jul 28 12:30:31.539115 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:31.539088    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ptmkj_771ef471-d88c-44c4-9506-99ff4adc16cf/extractor/0.log"
Jul 28 12:30:33.411612 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:33.411582    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dwgdc/perf-node-gather-daemonset-mn9tl"
Jul 28 12:30:34.199576 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:34.199541    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6954b9867f-tzmg8_c24e3e82-e29a-4a5a-b3f4-d09cb354139f/manager/0.log"
Jul 28 12:30:34.542308 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:34.540602    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6755d7577-jfn6z_624b4bc4-e9f3-447b-b96d-2a5e35dc4d60/manager/0.log"
Jul 28 12:30:34.560990 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:34.560950    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6755d7577-jfn6z_624b4bc4-e9f3-447b-b96d-2a5e35dc4d60/copy-manifests/0.log"
Jul 28 12:30:36.033223 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:36.033135    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65bf6bb88c-thsnz_5e2b9826-b0f5-4409-836d-c719892fc096/manager/0.log"
Jul 28 12:30:36.156376 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:36.156338    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-v48hg_6f95e161-5c34-4fce-915f-c37e57a514c9/discovery/0.log"
Jul 28 12:30:36.190366 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:36.190336    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8489dc4db9-rbztv_93386518-d4ed-4ceb-8706-e8daf0821ff9/kube-auth-proxy/0.log"
Jul 28 12:30:37.908062 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:37.907959    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-v48hg_6f95e161-5c34-4fce-915f-c37e57a514c9/discovery/0.log"
Jul 28 12:30:37.939249 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:37.939193    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8489dc4db9-rbztv_93386518-d4ed-4ceb-8706-e8daf0821ff9/kube-auth-proxy/0.log"
Jul 28 12:30:41.233648 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:41.233617    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-2knl7_c0f29548-3bd8-43c7-a768-0c848ac13d25/migrator/0.log"
Jul 28 12:30:41.256473 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:41.256444    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-2knl7_c0f29548-3bd8-43c7-a768-0c848ac13d25/graceful-termination/0.log"
Jul 28 12:30:41.675059 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:41.674943    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr_1b3487ae-02cf-4d41-83c1-f7bfa9e875fa/kube-storage-version-migrator-operator/1.log"
Jul 28 12:30:41.675846 ip-10-0-130-140 kubenswrapper[2552]: I0728 12:30:41.675826    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cjlnr_1b3487ae-02cf-4d41-83c1-f7bfa9e875fa/kube-storage-version-migrator-operator/0.log"