Jul 28 19:33:06.496324 ip-10-0-130-70 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:33:06.496337 ip-10-0-130-70 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:33:06.496347 ip-10-0-130-70 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:33:06.496714 ip-10-0-130-70 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:33:16.732335 ip-10-0-130-70 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 19:33:16.732353 ip-10-0-130-70 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 26a72661841c4745b0c2975e1e0159f8 --
Jul 28 19:35:14.568281 ip-10-0-130-70 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:35:14.958999 ip-10-0-130-70 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:35:14.958999 ip-10-0-130-70 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:35:14.958999 ip-10-0-130-70 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:35:14.958999 ip-10-0-130-70 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:35:14.958999 ip-10-0-130-70 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:35:14.961491 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.961369    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:35:14.965728 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965709    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.965728 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965725    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.965728 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965730    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.965728 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965734    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965739    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965744    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965751    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965757    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965760    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965764    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965776    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965780    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965784    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965788    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965792    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965796    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965800    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965804    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965808    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965812    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965817    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965821    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965825    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.965983 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965829    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965833    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965838    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965842    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965846    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965850    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965854    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965859    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965865    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965869    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965874    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965878    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965882    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965886    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965892    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965898    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965902    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965907    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965911    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965915    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.966791 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965920    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965924    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965928    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965935    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965957    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965962    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965966    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965971    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965975    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965979    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965983    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965987    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965991    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965995    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.965999    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966004    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966008    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966014    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966021    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.967690 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966026    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966031    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966035    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966039    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966044    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966048    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966052    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966058    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966063    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966068    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966072    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966077    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966081    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966085    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966089    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966094    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966098    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966103    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966111    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.968502 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966117    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966122    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966126    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966131    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966135    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966140    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966144    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966149    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966155    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966159    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966163    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966167    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966171    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966175    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966180    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966184    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966188    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.966193    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967354    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967365    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.969323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967373    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967378    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967383    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967388    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967393    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967397    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967401    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967405    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967410    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967414    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967419    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967423    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967427    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967432    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967436    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967440    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967444    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967448    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967452    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967456    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.970180 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967461    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967465    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967469    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967473    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967479    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967483    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967487    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967492    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967496    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967501    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967506    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967510    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967517    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967521    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967525    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967529    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967534    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967539    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967543    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.970878 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967547    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967551    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967555    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967560    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967564    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967568    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967572    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967576    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967581    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967585    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967589    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967593    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967598    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967602    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967606    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967610    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967614    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967618    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967624    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967628    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.971573 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967632    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967639    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967645    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967650    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967657    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967661    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967666    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967671    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967675    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967679    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967685    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967689    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967693    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967697    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967701    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967705    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967709    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967714    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967718    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.972174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967722    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967726    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967731    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967735    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967739    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967744    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967750    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967755    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967759    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967764    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967770    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967774    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967779    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967783    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967788    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967792    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967797    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967801    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.967805    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.972641 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968407    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968417    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968424    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968429    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968435    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968440    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968444    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968451    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968456    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968461    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968466    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968473    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968477    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968481    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968486    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968490    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968494    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968498    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.973212 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968502    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968506    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968510    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968514    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968518    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968523    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968527    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968531    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968535    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968540    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968544    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968549    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968553    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968558    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968562    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968566    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968570    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968574    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968578    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968583    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.973658 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968588    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968592    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968596    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968600    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968604    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968608    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968613    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968618    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968623    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968627    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968631    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968637    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968641    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968645    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968649    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968654    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968658    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968662    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968667    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968671    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.974174 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968675    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968680    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968684    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968689    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968694    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968698    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968702    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968706    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968711    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968715    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968719    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968724    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968729    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968733    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968737    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968741    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968745    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968750    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968754    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968760    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.974657 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968764    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968768    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968772    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968776    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968780    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968785    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968789    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968794    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968798    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968802    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968806    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968810    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968814    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968822    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968826    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968831    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968835    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968839    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968843    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.975169 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968848    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.968852    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.968977    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.968989    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.968997    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969004    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969012    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969017    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969023    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969030    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969035    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969039    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969045    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969050    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969055    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969060    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969065    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969070    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969074    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969079    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969086    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969091    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969096    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969102    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:35:14.975618 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969107    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969113    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969121    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969126    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969132    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969137    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969141    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969146    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969151    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969156    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969162    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969166    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969171    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969176    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969181    2575 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969187    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969192    2575 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969197    2575 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969202    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969207    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969211    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969225    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969229    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969234    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969239    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:35:14.976263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969244    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969249    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969254    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969258    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969263    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969268    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969274    2575 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969280    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969284    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969291    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969296    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969301    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969306    2575 flags.go:64] FLAG: --help="false"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969311    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-130-70.ec2.internal"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969316    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969320    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969325    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969330    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969336    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969340    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969344    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969348    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969353    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:35:14.976858 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969359    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969364    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969369    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969374    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969378    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969384    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969388    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969393    2575 flags.go:64] FLAG: --lock-file=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969398    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969403    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969407    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969416    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969421    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969425    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969430    2575 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969436    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969442    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969447    2575 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969453    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969460    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969464    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969471    2575 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969476    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969481    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969486    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:35:14.977486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969490    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969495    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969500    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969505    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969516    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969521    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969526    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969531    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969537    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969545    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969550    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969555    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969560    2575 flags.go:64] FLAG: --port="10250"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969565    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969570    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08c8cbe71625809dc"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969575    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969580    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969584    2575 flags.go:64] FLAG: --register-node="true"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969589    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969595    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969600    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969604    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969609    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969617    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969622    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:35:14.978120 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969628    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969633    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969638    2575 flags.go:64] FLAG: --runonce="false"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969643    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969648    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969653    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969657    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969662    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969667    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969671    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969676    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969681    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969686    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969690    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969695    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969700    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969705    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969710    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969719    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969724    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969729    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969735    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969740    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969744    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969748    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:35:14.978726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969753    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969758    2575 flags.go:64] FLAG: --v="2"
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969764    2575 flags.go:64] FLAG: --version="false"
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969770    2575 flags.go:64] FLAG: --vmodule=""
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969776    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.969783    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.969980    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.969990    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.969995    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970000    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970004    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970008    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970012    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970016    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970021    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970025    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970029    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970033    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970037    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970042    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970046    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.979358 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970050    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970054    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970058    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970063    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970067    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970072    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970076    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970080    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970085    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970089    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970093    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970097    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970101    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970106    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970110    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970114    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970120    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970124    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970130    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970134    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.979865 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970138    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970143    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970148    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970152    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970156    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970161    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970167    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970173    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970177    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970182    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970186    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970189    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970193    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970197    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970201    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970205    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970210    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970215    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970220    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970224    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.980378 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970228    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970232    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970237    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970241    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970245    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970250    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970254    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970258    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970264    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970268    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970274    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970278    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970282    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970286    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970290    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970295    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970300    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970304    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970309    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970313    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.980886 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970317    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970321    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970325    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970330    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970334    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970337    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970342    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970346    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970350    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970355    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970359    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970364    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970370    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970376    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970381    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970409    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970416    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970421    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970426    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.981380 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970431    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970438    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970443    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970451    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.970456    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.970469    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.977982    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.977996    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.978785    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.978791    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978855    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978861    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978864    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978867    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.981830 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978870    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978873    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978876    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978879    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978882    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978885    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978888    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978890    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978893    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978896    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978899    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978901    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978904    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978908    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978911    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978915    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978917    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978920    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978923    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978925    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.982199 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978928    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978930    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978933    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978936    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978949    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978954    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978957    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978960    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978962    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978965    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978967    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978970    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978973    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978975    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978978    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978980    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978983    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978985    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978988    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978990    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.982674 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978993    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978996    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.978999    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979001    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979004    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979006    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979009    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979012    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979014    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979017    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979019    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979022    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979024    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979027    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979029    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979032    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979034    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979038    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979042    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979044    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.983245 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979047    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979050    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979054    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979057    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979060    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979063    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979066    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979069    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979071    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979074    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979076    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979079    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979081    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979083    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979086    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979088    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979091    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979093    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979097    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.983740 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979101    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979104    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979107    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979109    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979112    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979114    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979118    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979120    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979122    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979125    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979128    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979131    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979134    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979137    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979139    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.984241 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979142    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.979147    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979274    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979280    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979283    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979286    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979289    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979291    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979294    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979296    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979299    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979302    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979305    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979307    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:35:14.984600 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979310    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979312    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979315    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979317    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979320    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979323    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979326    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979328    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979332    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979336    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979339    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979341    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979344    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979347    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979351    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979353    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979356    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979359    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979362    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:35:14.984958 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979365    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979368    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979370    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979373    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979376    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979378    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979381    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979383    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979386    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979388    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979390    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979393    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979395    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979398    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979400    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979403    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979405    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979408    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979410    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979413    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:35:14.985417 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979415    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979418    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979420    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979423    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979425    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979428    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979430    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979433    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979436    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979438    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979442    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979446    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979449    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979452    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979454    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979457    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979459    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979462    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979465    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:35:14.985895 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979467    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979470    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979473    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979476    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979478    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979481    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979483    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979486    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979488    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979491    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979493    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979496    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979498    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979501    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979504    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979507    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979511    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979513    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979516    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979519    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:35:14.986448 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979522    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979524    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979527    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979529    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979532    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979535    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979538    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979540    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:14.979543    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.979547    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.979644    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.981618    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.983038    2575 server.go:1019] "Starting client certificate rotation"
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.983141    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:35:14.986951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.983202    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:35:14.987294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.986869    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:35:14.992302 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:14.992276    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:35:15.005481 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.005461    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:35:15.007437 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.007422    2575 log.go:25] "Validated CRI v1 image API"
Jul 28 19:35:15.008268 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.008253    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:35:15.012927 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.012904    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:35:15.013886 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.013867    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aca44899-7744-412d-bb09-f341aebddb09:/dev/nvme0n1p3 bb2a5e87-9f76-483d-b71e-6c68104bbcc5:/dev/nvme0n1p4]
Jul 28 19:35:15.013925 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.013887    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:35:15.018009 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.017899    2575 manager.go:216] Machine: {Timestamp:2026-07-28 19:35:15.017445486 +0000 UTC m=+0.334111186 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104499 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2890ad22965bbc9cf17e4393be6840 SystemUUID:ec2890ad-2296-5bbc-9cf1-7e4393be6840 BootID:26a72661-841c-4745-b0c2-975e1e0159f8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:08:a8:3f:22:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:08:a8:3f:22:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:50:97:5a:a2:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:35:15.018009 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.018004    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:35:15.018113 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.018072    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:35:15.021048 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021023    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:35:15.021175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021051    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-70.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:35:15.021719 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021710    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:35:15.021755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021721    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:35:15.021755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021728    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:35:15.021755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.021739    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:35:15.022474 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.022464    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:35:15.024228 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024216    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:35:15.024344 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024335    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:35:15.024374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024363    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:35:15.024374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024373    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:35:15.024430 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024387    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:35:15.024430 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024394    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:35:15.024430 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024403    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:35:15.024740 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024729    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:35:15.024771 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.024755    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:35:15.026745 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.026729    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:35:15.028592 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.028577    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:35:15.028653 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.028607    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:35:15.030125 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030113    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030130    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030136    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030141    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030150    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030156    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030163    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:35:15.030166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030168    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:35:15.030353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030175    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:35:15.030353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030180    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:35:15.030353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030188    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:35:15.030353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030193    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:35:15.030908 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030898    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:35:15.030908 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.030908    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:35:15.033892 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.033878    2575 server.go:1268] "Started kubelet"
Jul 28 19:35:15.038723 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.038554    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:35:15.039233 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.039191    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zkvkc"
Jul 28 19:35:15.039369 ip-10-0-130-70 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:35:15.039841 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.034190    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:35:15.039924 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.039874    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:35:15.040169 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.040040    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:35:15.040169 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.040090    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-70.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:35:15.040681 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.040597    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:35:15.042673 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.042624    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:35:15.042773 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.042756    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:35:15.044285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.044267    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:35:15.046693 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.045797    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-70.ec2.internal.18c68b2a0db54afb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-70.ec2.internal,UID:ip-10-0-130-70.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-70.ec2.internal,},FirstTimestamp:2026-07-28 19:35:15.033852667 +0000 UTC m=+0.350518366,LastTimestamp:2026-07-28 19:35:15.033852667 +0000 UTC m=+0.350518366,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-70.ec2.internal,}"
Jul 28 19:35:15.051190 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051175    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:35:15.051270 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051195    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:35:15.051330 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.051310    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:15.051745 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051723    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:35:15.051838 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051760    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:35:15.051838 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051723    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:35:15.051934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051896    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:35:15.051934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.051907    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:35:15.052075 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052058    2575 factory.go:55] Registering systemd factory
Jul 28 19:35:15.052134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052078    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:35:15.052297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052285    2575 factory.go:153] Registering CRI-O factory
Jul 28 19:35:15.052374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052299    2575 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:35:15.052374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052347    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:35:15.052374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052351    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zkvkc"
Jul 28 19:35:15.052374 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052370    2575 factory.go:103] Registering Raw factory
Jul 28 19:35:15.052555 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052383    2575 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:35:15.052843 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.052829    2575 manager.go:349] Starting recovery of all containers
Jul 28 19:35:15.054094 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.054065    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:35:15.054197 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.054176    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:35:15.062755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.062741    2575 manager.go:354] Recovery completed
Jul 28 19:35:15.067109 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.067096    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:15.068830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.068817    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:15.068882 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.068843    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:15.068882 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.068853    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:15.069359 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.069344    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:35:15.069359 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.069357    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:35:15.069468 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.069374    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:35:15.069468 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.069377    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:15.070081 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.070064    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:15.070147 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.070093    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:15.070147 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.070104    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:15.071037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.071026    2575 policy_none.go:49] "None policy: Start"
Jul 28 19:35:15.071080 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.071041    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:35:15.071080 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.071051    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:35:15.072143 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.072120    2575 policy_none.go:47] "Start"
Jul 28 19:35:15.116626 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.116481    2575 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:35:15.116718 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.116641    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:35:15.116718 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.116653    2575 server.go:87] "Starting device plugin registration server"
Jul 28 19:35:15.116899 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.116886    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:35:15.116975 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.116900    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:35:15.117040 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.117027    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:35:15.117174 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.117110    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:35:15.117174 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:15.117124    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:35:15.117621 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.117591    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:35:15.117695 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:15.117643    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.644954 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.644915    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:16.646546 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.646525    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:16.646654 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.646564    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:16.646654 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.646580    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:16.646654 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.646608    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.657097 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.657081    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.657186 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.657105    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-70.ec2.internal\": node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.679624 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.679604    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.687507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.687477    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:35:16.688700 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.688685    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:35:16.688779 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.688716    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:35:16.688779 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.688749    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:35:16.688872 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.688787    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:35:16.690761 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.690745    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:35:16.780130 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.780081    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.789350 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.789311    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"]
Jul 28 19:35:16.789418 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.789393    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:16.791495 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.791475    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:16.791604 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.791508    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:16.791604 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.791520    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:16.792877 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.792865    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:16.793016 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793001    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.793076 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793034    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:16.793665 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793648    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:16.793764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793678    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:16.793764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793691    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:16.793764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793698    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:16.793764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793717    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:16.793764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.793740    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:16.794929 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.794911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.795018 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.794942    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:35:16.795683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.795667    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:35:16.795683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.795693    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:35:16.795838 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.795702    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:35:16.821430 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.821405    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-70.ec2.internal\" not found" node="ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.825807 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.825789    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-70.ec2.internal\" not found" node="ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.880800 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.880773    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.881911 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.881893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.881987 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.881923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e33f5784a8667eeea92ef51af7a40e20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-70.ec2.internal\" (UID: \"e33f5784a8667eeea92ef51af7a40e20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.881987 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.881951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.980931 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:16.980842    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:16.983021 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.983079 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.983079 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e33f5784a8667eeea92ef51af7a40e20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-70.ec2.internal\" (UID: \"e33f5784a8667eeea92ef51af7a40e20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.983142 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e33f5784a8667eeea92ef51af7a40e20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-70.ec2.internal\" (UID: \"e33f5784a8667eeea92ef51af7a40e20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.983142 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:16.983200 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:16.983100    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f917223d7503166a38a92a8b80122252-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal\" (UID: \"f917223d7503166a38a92a8b80122252\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:17.081286 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.081262    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:17.124588 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.124566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:17.129593 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.129571    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:17.181794 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.181761    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:17.282288 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.282214    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:17.382780 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.382754    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-70.ec2.internal\" not found"
Jul 28 19:35:17.410060 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.410040    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:35:17.410447 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.410173    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:35:17.431582 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.431558    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:35:17.452553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.452534    2575 apiserver.go:52] "Watching apiserver"
Jul 28 19:35:17.460875 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.460856    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:35:17.461231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.461210    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-q6pj8","openshift-image-registry/node-ca-rvt4g","openshift-multus/multus-7w8b2","openshift-network-diagnostics/network-check-target-vr28t","kube-system/konnectivity-agent-7x85f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9","openshift-cluster-node-tuning-operator/tuned-8cmpf","openshift-multus/multus-additional-cni-plugins-q8zqz","openshift-multus/network-metrics-daemon-vmhbw","openshift-network-operator/iptables-alerter-vxp72","openshift-ovn-kubernetes/ovnkube-node-m9nxc","kube-system/global-pull-secret-syncer-cgvwx"]
Jul 28 19:35:17.462698 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.462682    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.463671 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.463654    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.464592 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.464572    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.464656 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.464641    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.464706 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.464677    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cv7g4\""
Jul 28 19:35:17.465480 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.465466    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.465566 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.465526    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.465566 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.465532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wtzh4\""
Jul 28 19:35:17.465685 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.465558    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:35:17.465886 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.465874    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.468104 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.467899    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k875c\""
Jul 28 19:35:17.468214 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.468126    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:35:17.468214 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.468176    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.468498 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.468479    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.468601 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.468520    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:35:17.469151 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.469125    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:17.469241 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.469213    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:17.470413 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.470396    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.471684 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.471669    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.472269 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.472253    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:35:17.472344 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.472295    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:35:17.472386 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.472254    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bgntw\""
Jul 28 19:35:17.472777 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.472760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.473702 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.473686    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mtw9j\""
Jul 28 19:35:17.473951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.473936    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:35:17.474043 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474029    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.474321 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474300    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.474418 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474405    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.474828 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474812    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.474974 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474960    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.475044 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.474983    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bvgpc\""
Jul 28 19:35:17.475310 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.475296    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.475408 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.475369    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:17.475738 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.475712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:35:17.476018 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.476005    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:35:17.476390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.476377    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdmdk\""
Jul 28 19:35:17.476639 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.476625    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.477988 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.477972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.478372 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.478354    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.478548 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.478534    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.478714 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.478702    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-glbbp\""
Jul 28 19:35:17.478804 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.478782    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:35:17.479522 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.479505    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:35:17.479605 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.479507    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:17.479605 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.479535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.479605 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.479584    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:17.480034 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480006    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:35:17.480117 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480091    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:35:17.480296 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480281    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:35:17.480371 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480301    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:35:17.480424 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v9rm7\""
Jul 28 19:35:17.480473 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480435    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:35:17.480543 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480516    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:35:17.480666 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.480650    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:35:17.484620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.484219    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:30:16 +0000 UTC" deadline="2028-05-10 15:42:01.169539328 +0000 UTC"
Jul 28 19:35:17.484620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.484251    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15644h6m43.685291067s"
Jul 28 19:35:17.485435 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8318056b-0056-4938-a330-db29a044c0d9-tmp-dir\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.485525 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-device-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.485525 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485475    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-host\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.485525 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485505    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/525ad781-ea58-44d6-a259-97a82edab4a2-host-slash\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.485682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-netns\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.485682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.485682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-sys-fs\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.485682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-cni-binary-copy\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.485682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-kubelet\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.485850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.485850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb9c0960-6f62-4fbb-838c-9f436afebb45-serviceca\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.485850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.485850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.485850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485837    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzmt7\" (UniqueName: \"kubernetes.io/projected/525ad781-ea58-44d6-a259-97a82edab4a2-kube-api-access-vzmt7\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.485992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-multus\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.485992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-hostroot\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.485992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-script-lib\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.485992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8318056b-0056-4938-a330-db29a044c0d9-hosts-file\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.485992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.485987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-systemd\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.486140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8767354-38e8-48ec-9349-3ea7db7c0258-konnectivity-ca\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.486140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-var-lib-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-kubelet-config\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.486140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mb7j\" (UniqueName: \"kubernetes.io/projected/b880c10f-e514-463f-b428-31ae6d72bcb8-kube-api-access-8mb7j\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-conf\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486189    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6v2j\" (UniqueName: \"kubernetes.io/projected/777f220c-e1cb-4826-a60f-5546aabb3a23-kube-api-access-d6v2j\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486211    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-systemd-units\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-netd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-config\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-kubernetes\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-run\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-tuned\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-cnibin\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-socket-dir-parent\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-modprobe-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-cnibin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8767354-38e8-48ec-9349-3ea7db7c0258-agent-certs\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-slash\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.486570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r747\" (UniqueName: \"kubernetes.io/projected/fb9c0960-6f62-4fbb-838c-9f436afebb45-kube-api-access-6r747\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-sys\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-tmp\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486669    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-k8s-cni-cncf-io\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-bin\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8zkp\" (UniqueName: \"kubernetes.io/projected/bfe093a0-5621-47a9-b0eb-1443a018c1f2-kube-api-access-v8zkp\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486784    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-bin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzpft\" (UniqueName: \"kubernetes.io/projected/79fbf702-a163-4d8c-a3db-5a981fece6d8-kube-api-access-pzpft\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486867    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-systemd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-lib-modules\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486897    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-etc-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-node-log\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487011 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.486958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-dbus\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-netns\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-registration-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487519    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4c5t\" (UniqueName: \"kubernetes.io/projected/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-kube-api-access-f4c5t\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487546    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-system-cni-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.487572 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487568    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-system-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487584    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-os-release\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-daemon-config\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkwvf\" (UniqueName: \"kubernetes.io/projected/8318056b-0056-4938-a330-db29a044c0d9-kube-api-access-lkwvf\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-var-lib-kubelet\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-os-release\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-multus-certs\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-kubelet\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487771    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-ovn\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487795    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-log-socket\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-env-overrides\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb9c0960-6f62-4fbb-838c-9f436afebb45-host\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.487873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487863    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-socket-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7sxl\" (UniqueName: \"kubernetes.io/projected/877d18f7-d586-468e-9c8b-03870444164e-kube-api-access-k7sxl\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/525ad781-ea58-44d6-a259-97a82edab4a2-iptables-alerter-script\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487933    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-conf-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.487982    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.488004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysconfig\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.488025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-etc-kubernetes\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.488312 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.488065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovn-node-metrics-cert\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.489095 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.489076    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:35:17.489169 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.489132    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"
Jul 28 19:35:17.489169 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.489082    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal"]
Jul 28 19:35:17.492101 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.492079    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:35:17.500644 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.500629    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:35:17.500750 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.500672    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal"]
Jul 28 19:35:17.520778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.520758    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ht7rs"
Jul 28 19:35:17.529227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.529207    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ht7rs"
Jul 28 19:35:17.574951 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.574929    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:35:17.588396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588377    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-host\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/525ad781-ea58-44d6-a259-97a82edab4a2-host-slash\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-netns\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-sys-fs\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-cni-binary-copy\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588492    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/525ad781-ea58-44d6-a259-97a82edab4a2-host-slash\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.588502 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588493    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-host\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-kubelet\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588505    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-netns\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-sys-fs\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb9c0960-6f62-4fbb-838c-9f436afebb45-serviceca\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-kubelet\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588581    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588591    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmt7\" (UniqueName: \"kubernetes.io/projected/525ad781-ea58-44d6-a259-97a82edab4a2-kube-api-access-vzmt7\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-multus\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-hostroot\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-script-lib\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.588863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8318056b-0056-4938-a330-db29a044c0d9-hosts-file\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-multus\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-systemd\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-hostroot\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8767354-38e8-48ec-9349-3ea7db7c0258-konnectivity-ca\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-var-lib-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588903    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-systemd\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588909    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-kubelet-config\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588860    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8318056b-0056-4938-a330-db29a044c0d9-hosts-file\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mb7j\" (UniqueName: \"kubernetes.io/projected/b880c10f-e514-463f-b428-31ae6d72bcb8-kube-api-access-8mb7j\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.588981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-conf\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6v2j\" (UniqueName: \"kubernetes.io/projected/777f220c-e1cb-4826-a60f-5546aabb3a23-kube-api-access-d6v2j\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-cni-binary-copy\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-systemd-units\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.589633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-netd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-config\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fb9c0960-6f62-4fbb-838c-9f436afebb45-serviceca\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-kubernetes\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-systemd-units\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589231    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-kubernetes\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-run\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-netd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-tuned\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589306    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-cnibin\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.589375    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-socket-dir-parent\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-modprobe-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysctl-conf\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.589440    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:18.089410667 +0000 UTC m=+1.978287632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589469    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-cnibin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.590456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8767354-38e8-48ec-9349-3ea7db7c0258-agent-certs\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-modprobe-d\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-cnibin\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-slash\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r747\" (UniqueName: \"kubernetes.io/projected/fb9c0960-6f62-4fbb-838c-9f436afebb45-kube-api-access-6r747\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-run\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-sys\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589642    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-tmp\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-k8s-cni-cncf-io\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589707    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-bin\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8zkp\" (UniqueName: \"kubernetes.io/projected/bfe093a0-5621-47a9-b0eb-1443a018c1f2-kube-api-access-v8zkp\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-socket-dir-parent\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589809    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-bin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzpft\" (UniqueName: \"kubernetes.io/projected/79fbf702-a163-4d8c-a3db-5a981fece6d8-kube-api-access-pzpft\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589848    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-sys\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-systemd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.591236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8767354-38e8-48ec-9349-3ea7db7c0258-konnectivity-ca\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-cnibin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589890    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-lib-modules\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.589948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-var-lib-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-lib-modules\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-etc-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-script-lib\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-etc-openvswitch\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590109    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovnkube-config\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590333    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-k8s-cni-cncf-io\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590369    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-node-log\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-systemd\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-cni-bin\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-var-lib-cni-bin\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592134 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-dbus\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590494    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-netns\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590495    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-node-log\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-slash\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590568    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-dbus\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590591    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590646    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-registration-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590682    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4c5t\" (UniqueName: \"kubernetes.io/projected/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-kube-api-access-f4c5t\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590741    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-registration-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-system-cni-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-kubelet-config\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-system-cni-dir\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-system-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-netns\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.592978 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-os-release\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590885    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-system-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-daemon-config\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-os-release\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.590969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkwvf\" (UniqueName: \"kubernetes.io/projected/8318056b-0056-4938-a330-db29a044c0d9-kube-api-access-lkwvf\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-var-lib-kubelet\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-os-release\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/877d18f7-d586-468e-9c8b-03870444164e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-multus-certs\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-host-run-multus-certs\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-kubelet\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-var-lib-kubelet\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-ovn\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-log-socket\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/877d18f7-d586-468e-9c8b-03870444164e-os-release\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-env-overrides\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591339    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-run-ovn\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb9c0960-6f62-4fbb-838c-9f436afebb45-host\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.593553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-socket-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-host-kubelet\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-daemon-config\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591474    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb9c0960-6f62-4fbb-838c-9f436afebb45-host\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7sxl\" (UniqueName: \"kubernetes.io/projected/877d18f7-d586-468e-9c8b-03870444164e-kube-api-access-k7sxl\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/525ad781-ea58-44d6-a259-97a82edab4a2-iptables-alerter-script\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-conf-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591591    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfe093a0-5621-47a9-b0eb-1443a018c1f2-log-socket\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysconfig\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591633    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-conf-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-etc-kubernetes\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovn-node-metrics-cert\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591690    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-multus-cni-dir\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8318056b-0056-4938-a330-db29a044c0d9-tmp-dir\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591773    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-device-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfe093a0-5621-47a9-b0eb-1443a018c1f2-env-overrides\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.594162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-socket-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b880c10f-e514-463f-b428-31ae6d72bcb8-device-dir\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fbf702-a163-4d8c-a3db-5a981fece6d8-etc-kubernetes\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.591878    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.591890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-sysconfig\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.591927    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:18.091910548 +0000 UTC m=+1.980787513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.592054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8318056b-0056-4938-a330-db29a044c0d9-tmp-dir\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.592222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/525ad781-ea58-44d6-a259-97a82edab4a2-iptables-alerter-script\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.593291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-etc-tuned\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.593424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8767354-38e8-48ec-9349-3ea7db7c0258-agent-certs\") pod \"konnectivity-agent-7x85f\" (UID: \"f8767354-38e8-48ec-9349-3ea7db7c0258\") " pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.593953    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-tmp\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.594696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.594190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfe093a0-5621-47a9-b0eb-1443a018c1f2-ovn-node-metrics-cert\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.597535 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.597517    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:17.597631 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.597542    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:17.597631 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.597554    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:17.597712 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:17.597646    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:18.09762434 +0000 UTC m=+1.986501291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:17.600109 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.600082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6v2j\" (UniqueName: \"kubernetes.io/projected/777f220c-e1cb-4826-a60f-5546aabb3a23-kube-api-access-d6v2j\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:17.600346 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.600326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzmt7\" (UniqueName: \"kubernetes.io/projected/525ad781-ea58-44d6-a259-97a82edab4a2-kube-api-access-vzmt7\") pod \"iptables-alerter-vxp72\" (UID: \"525ad781-ea58-44d6-a259-97a82edab4a2\") " pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.600694 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.600668    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mb7j\" (UniqueName: \"kubernetes.io/projected/b880c10f-e514-463f-b428-31ae6d72bcb8-kube-api-access-8mb7j\") pod \"aws-ebs-csi-driver-node-wwzn9\" (UID: \"b880c10f-e514-463f-b428-31ae6d72bcb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.602659 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.602585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzpft\" (UniqueName: \"kubernetes.io/projected/79fbf702-a163-4d8c-a3db-5a981fece6d8-kube-api-access-pzpft\") pod \"multus-7w8b2\" (UID: \"79fbf702-a163-4d8c-a3db-5a981fece6d8\") " pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.602830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.602816    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8zkp\" (UniqueName: \"kubernetes.io/projected/bfe093a0-5621-47a9-b0eb-1443a018c1f2-kube-api-access-v8zkp\") pod \"ovnkube-node-m9nxc\" (UID: \"bfe093a0-5621-47a9-b0eb-1443a018c1f2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.602977 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.602958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4c5t\" (UniqueName: \"kubernetes.io/projected/77d7f1e0-813c-4922-bb2c-400f3dfeed6b-kube-api-access-f4c5t\") pod \"tuned-8cmpf\" (UID: \"77d7f1e0-813c-4922-bb2c-400f3dfeed6b\") " pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.603195 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.603177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7sxl\" (UniqueName: \"kubernetes.io/projected/877d18f7-d586-468e-9c8b-03870444164e-kube-api-access-k7sxl\") pod \"multus-additional-cni-plugins-q8zqz\" (UID: \"877d18f7-d586-468e-9c8b-03870444164e\") " pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.606037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.606017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r747\" (UniqueName: \"kubernetes.io/projected/fb9c0960-6f62-4fbb-838c-9f436afebb45-kube-api-access-6r747\") pod \"node-ca-rvt4g\" (UID: \"fb9c0960-6f62-4fbb-838c-9f436afebb45\") " pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.607093 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.607071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkwvf\" (UniqueName: \"kubernetes.io/projected/8318056b-0056-4938-a330-db29a044c0d9-kube-api-access-lkwvf\") pod \"node-resolver-q6pj8\" (UID: \"8318056b-0056-4938-a330-db29a044c0d9\") " pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.614828 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.614813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8zqz"
Jul 28 19:35:17.632048 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.632026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vxp72"
Jul 28 19:35:17.638724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.638707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:17.643151 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.643114    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877d18f7_d586_468e_9c8b_03870444164e.slice/crio-0bb9beb4b98abe05140532fa2039a8d54662ff5943184707138d3c96e46810da WatchSource:0}: Error finding container 0bb9beb4b98abe05140532fa2039a8d54662ff5943184707138d3c96e46810da: Status 404 returned error can't find the container with id 0bb9beb4b98abe05140532fa2039a8d54662ff5943184707138d3c96e46810da
Jul 28 19:35:17.644824 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.644797    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf917223d7503166a38a92a8b80122252.slice/crio-006d484ca8dc7f4ddfcd827b5b13992272bd3df1fe335a1ebfa19095ff3a35af WatchSource:0}: Error finding container 006d484ca8dc7f4ddfcd827b5b13992272bd3df1fe335a1ebfa19095ff3a35af: Status 404 returned error can't find the container with id 006d484ca8dc7f4ddfcd827b5b13992272bd3df1fe335a1ebfa19095ff3a35af
Jul 28 19:35:17.647440 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.647424    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:35:17.648119 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.648094    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525ad781_ea58_44d6_a259_97a82edab4a2.slice/crio-e34bacaa2131f84a53f4f44bf3fe22fbd010e823d589cda5b426a66debcdbd39 WatchSource:0}: Error finding container e34bacaa2131f84a53f4f44bf3fe22fbd010e823d589cda5b426a66debcdbd39: Status 404 returned error can't find the container with id e34bacaa2131f84a53f4f44bf3fe22fbd010e823d589cda5b426a66debcdbd39
Jul 28 19:35:17.650662 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.650641    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe093a0_5621_47a9_b0eb_1443a018c1f2.slice/crio-3f17a76ac088848d420eb0e894c994b10d1d0b4be6d0cc5d6dbd3722f87aa4cb WatchSource:0}: Error finding container 3f17a76ac088848d420eb0e894c994b10d1d0b4be6d0cc5d6dbd3722f87aa4cb: Status 404 returned error can't find the container with id 3f17a76ac088848d420eb0e894c994b10d1d0b4be6d0cc5d6dbd3722f87aa4cb
Jul 28 19:35:17.692047 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.691998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerStarted","Data":"0bb9beb4b98abe05140532fa2039a8d54662ff5943184707138d3c96e46810da"}
Jul 28 19:35:17.693473 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.693453    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"3f17a76ac088848d420eb0e894c994b10d1d0b4be6d0cc5d6dbd3722f87aa4cb"}
Jul 28 19:35:17.694529 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.694512    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vxp72" event={"ID":"525ad781-ea58-44d6-a259-97a82edab4a2","Type":"ContainerStarted","Data":"e34bacaa2131f84a53f4f44bf3fe22fbd010e823d589cda5b426a66debcdbd39"}
Jul 28 19:35:17.695617 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.695599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal" event={"ID":"f917223d7503166a38a92a8b80122252","Type":"ContainerStarted","Data":"006d484ca8dc7f4ddfcd827b5b13992272bd3df1fe335a1ebfa19095ff3a35af"}
Jul 28 19:35:17.696570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.696554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal" event={"ID":"e33f5784a8667eeea92ef51af7a40e20","Type":"ContainerStarted","Data":"b02addc1c14621b148f8041d0b5d03b13bcf6ddb31ca38416a4640e5a19cd4eb"}
Jul 28 19:35:17.796977 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.796901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q6pj8"
Jul 28 19:35:17.804492 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.804465    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8318056b_0056_4938_a330_db29a044c0d9.slice/crio-3cee9c72e1fc2554488803c6d8ebfd401f398d44a58ea1a837bae79fe12add06 WatchSource:0}: Error finding container 3cee9c72e1fc2554488803c6d8ebfd401f398d44a58ea1a837bae79fe12add06: Status 404 returned error can't find the container with id 3cee9c72e1fc2554488803c6d8ebfd401f398d44a58ea1a837bae79fe12add06
Jul 28 19:35:17.815893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.815871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvt4g"
Jul 28 19:35:17.823323 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.823299    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9c0960_6f62_4fbb_838c_9f436afebb45.slice/crio-27514daf11761a43e50022d75e7cc48c19fa1e48b365b700188f1f6e0ca98ef2 WatchSource:0}: Error finding container 27514daf11761a43e50022d75e7cc48c19fa1e48b365b700188f1f6e0ca98ef2: Status 404 returned error can't find the container with id 27514daf11761a43e50022d75e7cc48c19fa1e48b365b700188f1f6e0ca98ef2
Jul 28 19:35:17.836481 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.836463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7w8b2"
Jul 28 19:35:17.843607 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.843588    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:17.843920 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.843899    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fbf702_a163_4d8c_a3db_5a981fece6d8.slice/crio-be27bad79c4dfb074ae2e1e827d1e4c981a65daf2e9928749ef56b9ffce5c0d7 WatchSource:0}: Error finding container be27bad79c4dfb074ae2e1e827d1e4c981a65daf2e9928749ef56b9ffce5c0d7: Status 404 returned error can't find the container with id be27bad79c4dfb074ae2e1e827d1e4c981a65daf2e9928749ef56b9ffce5c0d7
Jul 28 19:35:17.851615 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.851591    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8767354_38e8_48ec_9349_3ea7db7c0258.slice/crio-4f856d10b8334d79aa907d3826ffdeb7f3669f193eb6b7efa092a6c3d4a6900f WatchSource:0}: Error finding container 4f856d10b8334d79aa907d3826ffdeb7f3669f193eb6b7efa092a6c3d4a6900f: Status 404 returned error can't find the container with id 4f856d10b8334d79aa907d3826ffdeb7f3669f193eb6b7efa092a6c3d4a6900f
Jul 28 19:35:17.860826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.860807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9"
Jul 28 19:35:17.867592 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.867569    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb880c10f_e514_463f_b428_31ae6d72bcb8.slice/crio-13b978b892abbc3820b46ee43be7378b928013d82e63d98f733a749a7d82611a WatchSource:0}: Error finding container 13b978b892abbc3820b46ee43be7378b928013d82e63d98f733a749a7d82611a: Status 404 returned error can't find the container with id 13b978b892abbc3820b46ee43be7378b928013d82e63d98f733a749a7d82611a
Jul 28 19:35:17.881369 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.881344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8cmpf"
Jul 28 19:35:17.888012 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:35:17.887993    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d7f1e0_813c_4922_bb2c_400f3dfeed6b.slice/crio-c0a219f3a3b4e5d76b58e13f7fb02ba0f590b14635b0a262559e18b713c4b458 WatchSource:0}: Error finding container c0a219f3a3b4e5d76b58e13f7fb02ba0f590b14635b0a262559e18b713c4b458: Status 404 returned error can't find the container with id c0a219f3a3b4e5d76b58e13f7fb02ba0f590b14635b0a262559e18b713c4b458
Jul 28 19:35:17.934704 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:17.934680    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:35:18.095508 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.094923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:18.095508 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.094990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:18.095508 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.095136    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:18.095508 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.095191    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:19.095173885 +0000 UTC m=+2.984050840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:18.095508 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.095503    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:18.095943 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.095559    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:19.095544554 +0000 UTC m=+2.984421516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:18.196402 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.195564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:18.196402 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.195711    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:18.196402 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.195746    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:18.196402 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.195761    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:18.196402 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:18.195815    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:19.195797947 +0000 UTC m=+3.084674900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:18.531746 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.531585    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:30:17 +0000 UTC" deadline="2028-01-02 00:53:20.496198854 +0000 UTC"
Jul 28 19:35:18.531746 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.531631    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12533h18m1.964573891s"
Jul 28 19:35:18.720436 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.720380    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvt4g" event={"ID":"fb9c0960-6f62-4fbb-838c-9f436afebb45","Type":"ContainerStarted","Data":"27514daf11761a43e50022d75e7cc48c19fa1e48b365b700188f1f6e0ca98ef2"}
Jul 28 19:35:18.737568 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.737529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7x85f" event={"ID":"f8767354-38e8-48ec-9349-3ea7db7c0258","Type":"ContainerStarted","Data":"4f856d10b8334d79aa907d3826ffdeb7f3669f193eb6b7efa092a6c3d4a6900f"}
Jul 28 19:35:18.755265 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.755230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q6pj8" event={"ID":"8318056b-0056-4938-a330-db29a044c0d9","Type":"ContainerStarted","Data":"3cee9c72e1fc2554488803c6d8ebfd401f398d44a58ea1a837bae79fe12add06"}
Jul 28 19:35:18.758423 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.758312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8cmpf" event={"ID":"77d7f1e0-813c-4922-bb2c-400f3dfeed6b","Type":"ContainerStarted","Data":"c0a219f3a3b4e5d76b58e13f7fb02ba0f590b14635b0a262559e18b713c4b458"}
Jul 28 19:35:18.774674 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.774638    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9" event={"ID":"b880c10f-e514-463f-b428-31ae6d72bcb8","Type":"ContainerStarted","Data":"13b978b892abbc3820b46ee43be7378b928013d82e63d98f733a749a7d82611a"}
Jul 28 19:35:18.812486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.812375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7w8b2" event={"ID":"79fbf702-a163-4d8c-a3db-5a981fece6d8","Type":"ContainerStarted","Data":"be27bad79c4dfb074ae2e1e827d1e4c981a65daf2e9928749ef56b9ffce5c0d7"}
Jul 28 19:35:18.942924 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:18.942702    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:35:19.110995 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.110962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:19.111185 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.111027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:19.111185 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.111172    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:19.111328 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.111228    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:21.111208705 +0000 UTC m=+5.000085656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:19.111637 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.111618    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:19.111720 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.111674    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:21.111659058 +0000 UTC m=+5.000536008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:19.212375 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.211819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:19.212375 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.211973    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:19.212375 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.211993    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:19.212375 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.212006    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:19.212375 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.212059    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:21.212042007 +0000 UTC m=+5.100918958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:19.532924 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.532761    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:30:17 +0000 UTC" deadline="2028-03-13 09:25:18.824595256 +0000 UTC"
Jul 28 19:35:19.532924 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.532825    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14245h49m59.291774821s"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.689544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.689684    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.690115    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.690220    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:19.690284    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:19.690417 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:19.690372    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:21.127179 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.127142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:21.127627 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.127216    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:21.127627 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.127377    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:21.127627 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.127442    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:25.127422991 +0000 UTC m=+9.016299944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:21.127972 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.127848    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:21.127972 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.127908    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:25.127892091 +0000 UTC m=+9.016769040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:21.227787 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.227747    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:21.227967 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.227914    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:21.227967 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.227934    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:21.227967 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.227946    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:21.228082 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.228049    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:25.228030388 +0000 UTC m=+9.116907337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.689260    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.689392    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.689773    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.689863    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:21.689910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:21.690029 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:21.689988    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:23.689379    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:23.689411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:23.689412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:23.689495    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:23.689586    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:23.689769 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:23.689654    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:25.161113 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.160946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:25.161113 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.161072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:25.161113 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.161080    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:25.161708 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.161144    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:33.161124722 +0000 UTC m=+17.050001694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:25.161708 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.161163    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:25.161708 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.161196    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:33.161186075 +0000 UTC m=+17.050063024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:25.261717 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.261679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:25.261927 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.261871    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:25.261927 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.261902    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:25.261927 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.261917    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:25.262070 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.261982    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:33.261962952 +0000 UTC m=+17.150839903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:25.689513 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.689172    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:25.689513 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.689205    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:25.689513 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.689302    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:25.689841 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.689749    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:25.689841 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:25.689819    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:25.689948 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:25.689909    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:27.689651 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:27.689615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:27.690094 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:27.689629    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:27.690094 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:27.689762    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:27.690094 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:27.689631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:27.690094 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:27.689880    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:27.690094 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:27.690000    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:29.688967 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:29.688934    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:29.689395 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:29.688982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:29.689395 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:29.688982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:29.689395 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:29.689052    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:29.689395 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:29.689162    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:29.689395 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:29.689257    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:31.689340 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:31.689298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:31.689830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:31.689298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:31.689830 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:31.689413    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:31.689830 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:31.689531    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:31.689830 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:31.689313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:31.689830 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:31.689639    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:33.224543 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.224501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:33.224998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.224647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:33.224998 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.224669    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:33.224998 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.224829    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:35:49.22480381 +0000 UTC m=+33.113680766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:33.224998 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.224743    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:33.224998 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.224893    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:49.224874802 +0000 UTC m=+33.113751755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:33.325539 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.325500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:33.325744 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.325680    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:33.325744 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.325702    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:33.325744 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.325712    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:33.325911 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.325787    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:49.325772676 +0000 UTC m=+33.214649624 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:33.689309 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.689104    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:33.689506 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.689104    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:33.689506 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.689407    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:33.689506 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:33.689165    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:33.689506 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.689489    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:33.689687 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:33.689574    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:35.689872 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:35.689841    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:35.689872 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:35.689876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:35.690443 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:35.689959    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:35.690443 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:35.689976    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:35.690443 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:35.690024    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:35.690443 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:35.690090    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:37.689918 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.689885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:37.690315 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.689963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:37.690315 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.690061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:37.690315 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:37.690080    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:37.690315 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:37.690136    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:37.690315 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:37.690198    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:37.855855 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.855783    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8cmpf" event={"ID":"77d7f1e0-813c-4922-bb2c-400f3dfeed6b","Type":"ContainerStarted","Data":"af881b7c7e8865d294967c96461d30309f2ede29d046fb2c4d1e0aebbccd170e"}
Jul 28 19:35:37.857969 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.857943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7w8b2" event={"ID":"79fbf702-a163-4d8c-a3db-5a981fece6d8","Type":"ContainerStarted","Data":"efd13ac67119bddb87d6cb1a5409a1691590e0bfa6a4def8eff4fbc1d2f61923"}
Jul 28 19:35:37.859448 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.859212    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal" event={"ID":"e33f5784a8667eeea92ef51af7a40e20","Type":"ContainerStarted","Data":"e14608df696152c3886160cc6987a948448015238faba60e4fa1d3012750a007"}
Jul 28 19:35:37.873342 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.873302    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8cmpf" podStartSLOduration=2.744585956 podStartE2EDuration="21.873289607s" totalImagesPullingTime="19.128703651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:37.872802726 +0000 UTC m=+21.761679694" watchObservedRunningTime="2026-07-28 19:35:37.873289607 +0000 UTC m=+21.762166577"
Jul 28 19:35:37.892767 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.892116    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7w8b2" podStartSLOduration=2.712960981 podStartE2EDuration="21.892095045s" totalImagesPullingTime="19.179134064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:37.88976336 +0000 UTC m=+21.778640330" watchObservedRunningTime="2026-07-28 19:35:37.892095045 +0000 UTC m=+21.780972016"
Jul 28 19:35:37.907472 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:37.907435    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-70.ec2.internal" podStartSLOduration=20.907421715 podStartE2EDuration="20.907421715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:37.906907731 +0000 UTC m=+21.795784701" watchObservedRunningTime="2026-07-28 19:35:37.907421715 +0000 UTC m=+21.796298686"
Jul 28 19:35:38.863679 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.863383    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.863981    2575 generic.go:356] "Generic (PLEG): container finished" podID="bfe093a0-5621-47a9-b0eb-1443a018c1f2" containerID="398d28bd9ed526c413966ba94aa2a8c9c4afeb95c72f079745497d61de98823f" exitCode=1
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"f5983b42c424b227fc84dba41db6919ba962e546a59b40db7a8b5015865e716b"}
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"4075cf75ea4f2c11c9cd53a48e56109bc6d1194c2a3039c3505f5e9384542502"}
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"a413135557d194498baa4a99f0d9e0ce2b21333694782918a35808270029147c"}
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"8ee6ccf81417e2484360d1f05ebaa23314bd4d0ebd95346c6a42414ce47d6e59"}
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerDied","Data":"398d28bd9ed526c413966ba94aa2a8c9c4afeb95c72f079745497d61de98823f"}
Jul 28 19:35:38.864464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.864277    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"a28b0528d271cfa6fd4415e702565a23d055508193e19df062b497d581e14f9a"}
Jul 28 19:35:38.865528 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.865505    2575 generic.go:356] "Generic (PLEG): container finished" podID="f917223d7503166a38a92a8b80122252" containerID="6302ed660b1a09f87f51c96e5865a17fc8b3e43213bcbc1e4f06173082262b0f" exitCode=0
Jul 28 19:35:38.865634 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.865588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal" event={"ID":"f917223d7503166a38a92a8b80122252","Type":"ContainerDied","Data":"6302ed660b1a09f87f51c96e5865a17fc8b3e43213bcbc1e4f06173082262b0f"}
Jul 28 19:35:38.867042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.867024    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9" event={"ID":"b880c10f-e514-463f-b428-31ae6d72bcb8","Type":"ContainerStarted","Data":"96a78a30e8a18ac7f85b1a859d84b49d4296ee1c1408d8e72072c0726225e335"}
Jul 28 19:35:38.868549 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.868522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvt4g" event={"ID":"fb9c0960-6f62-4fbb-838c-9f436afebb45","Type":"ContainerStarted","Data":"415890f941c6d94ae83044079d4a9bcc4b3df3c2b0d3be24075c34e2a1ca951e"}
Jul 28 19:35:38.869776 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.869756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vxp72" event={"ID":"525ad781-ea58-44d6-a259-97a82edab4a2","Type":"ContainerStarted","Data":"988d5c036521cc1f5020fd7da078e951148a7d501c01de7b9a5b1a5fcfc3122d"}
Jul 28 19:35:38.871116 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.871095    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="28aa40848fc2e9715774667e77dab2853dedbaa4b086e616c980ec7ea1d53131" exitCode=0
Jul 28 19:35:38.871217 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.871172    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"28aa40848fc2e9715774667e77dab2853dedbaa4b086e616c980ec7ea1d53131"}
Jul 28 19:35:38.872523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.872500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7x85f" event={"ID":"f8767354-38e8-48ec-9349-3ea7db7c0258","Type":"ContainerStarted","Data":"a3d3a6d9ef551020b2d1d86d811b20720a0b322e3ffb242f21bae4ab59006f4f"}
Jul 28 19:35:38.873891 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.873871    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q6pj8" event={"ID":"8318056b-0056-4938-a330-db29a044c0d9","Type":"ContainerStarted","Data":"715fd8bc490417f711ea00d4fc688e2cb86126da4098ed361f27b238bc54db67"}
Jul 28 19:35:38.911540 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.911492    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7x85f" podStartSLOduration=3.747566515 podStartE2EDuration="22.911476142s" totalImagesPullingTime="19.163909627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.911024204 +0000 UTC m=+22.799901177" watchObservedRunningTime="2026-07-28 19:35:38.911476142 +0000 UTC m=+22.800353110"
Jul 28 19:35:38.957037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:38.956997    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vxp72" podStartSLOduration=3.590779652 podStartE2EDuration="22.956984336s" totalImagesPullingTime="19.366204684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.956669162 +0000 UTC m=+22.845546136" watchObservedRunningTime="2026-07-28 19:35:38.956984336 +0000 UTC m=+22.845861307"
Jul 28 19:35:39.014788 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.014745    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q6pj8" podStartSLOduration=3.796571114 podStartE2EDuration="23.014715941s" totalImagesPullingTime="19.218144827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.992520634 +0000 UTC m=+22.881397604" watchObservedRunningTime="2026-07-28 19:35:39.014715941 +0000 UTC m=+22.903592950"
Jul 28 19:35:39.227878 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.227852    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:35:39.558460 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.558333    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:35:39.227873166Z","UUID":"9b863b42-5d5d-4566-9ef9-8647309bad22","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:35:39.560046 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.560015    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:35:39.560174 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.560058    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:35:39.689271 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.689198    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:39.689504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.689319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:39.689504 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:39.689330    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:39.689504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.689381    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:39.689504 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:39.689473    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:39.689787 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:39.689559    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:39.878313 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.878277    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal" event={"ID":"f917223d7503166a38a92a8b80122252","Type":"ContainerStarted","Data":"6eca41a9a7f0f05cc067eaced0cff2b6f2abca84b9ed945fd4da65f61cd19de4"}
Jul 28 19:35:39.880396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.880359    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9" event={"ID":"b880c10f-e514-463f-b428-31ae6d72bcb8","Type":"ContainerStarted","Data":"c431e299b632eded10f54843bb5eb7c775ac8520317a4238eb24844b4b21971f"}
Jul 28 19:35:39.893562 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:39.893520    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rvt4g" podStartSLOduration=4.7025383309999995 podStartE2EDuration="23.893505331s" totalImagesPullingTime="19.190967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:39.01626826 +0000 UTC m=+22.905145230" watchObservedRunningTime="2026-07-28 19:35:39.893505331 +0000 UTC m=+23.782382301"
Jul 28 19:35:40.139296 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.139214    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q6pj8_8318056b-0056-4938-a330-db29a044c0d9/dns-node-resolver/0.log"
Jul 28 19:35:40.886155 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.885993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:35:40.886591 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.886567    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"77e13fa35ccfc8583c2bb7053f536956043398210c4872d0694f696f6adf4dbd"}
Jul 28 19:35:40.889111 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.889071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9" event={"ID":"b880c10f-e514-463f-b428-31ae6d72bcb8","Type":"ContainerStarted","Data":"bb9e626ef32c03ad90a2f8a987ceb5a6b806b2ff3dcb6406d1af7a57f611fd79"}
Jul 28 19:35:40.906037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.905975    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-70.ec2.internal" podStartSLOduration=23.905962144 podStartE2EDuration="23.905962144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:39.893056467 +0000 UTC m=+23.781933437" watchObservedRunningTime="2026-07-28 19:35:40.905962144 +0000 UTC m=+24.794839116"
Jul 28 19:35:40.906160 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:40.906085    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwzn9" podStartSLOduration=3.412266921 podStartE2EDuration="24.906078368s" totalImagesPullingTime="21.493811447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:40.905475108 +0000 UTC m=+24.794352080" watchObservedRunningTime="2026-07-28 19:35:40.906078368 +0000 UTC m=+24.794955398"
Jul 28 19:35:41.110100 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:41.110075    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvt4g_fb9c0960-6f62-4fbb-838c-9f436afebb45/node-ca/0.log"
Jul 28 19:35:41.689585 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:41.689559    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:41.689795 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:41.689559    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:41.689795 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:41.689679    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:41.689795 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:41.689691    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:41.689972 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:41.689803    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:41.689972 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:41.689898    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:42.898127 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.897980    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:35:42.899451 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.898374    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"e4fe5dd0b6614c8e1950b62a7a41197916f649657741147e599380f0701c3b91"}
Jul 28 19:35:42.899451 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.898665    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:42.899451 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.898758    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:42.899451 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.898809    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:42.899451 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.898881    2575 scope.go:117] "RemoveContainer" containerID="398d28bd9ed526c413966ba94aa2a8c9c4afeb95c72f079745497d61de98823f"
Jul 28 19:35:42.914378 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.914348    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:42.915353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:42.915332    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:35:43.262119 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.261855    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:43.262414 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.262398    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:43.689932 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.689901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:43.690160 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.689944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:43.690160 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.690003    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:43.690160 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:43.690021    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:43.690160 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:43.690100    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:43.690308 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:43.690157    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:43.902574 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.902543    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="5afaff06c2365433060c9bb454dcc2e0bb835ea4a9344ab80fa9d4d967806d8a" exitCode=0
Jul 28 19:35:43.903152 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.902608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"5afaff06c2365433060c9bb454dcc2e0bb835ea4a9344ab80fa9d4d967806d8a"}
Jul 28 19:35:43.906482 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.906466    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:35:43.906826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.906805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" event={"ID":"bfe093a0-5621-47a9-b0eb-1443a018c1f2","Type":"ContainerStarted","Data":"26b996cf6d5b507c69d9dc835818cb171f3e89b61d9ff2c6e11f19117c746c48"}
Jul 28 19:35:43.906997 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.906984    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:43.907423 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.907407    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7x85f"
Jul 28 19:35:43.971052 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:43.970983    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc" podStartSLOduration=7.809893435 podStartE2EDuration="27.970969323s" totalImagesPullingTime="20.161075888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:43.970653758 +0000 UTC m=+27.859530729" watchObservedRunningTime="2026-07-28 19:35:43.970969323 +0000 UTC m=+27.859846295"
Jul 28 19:35:45.689850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:45.689819    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:45.690244 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:45.689858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:45.690244 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:45.689907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:45.690244 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:45.689920    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:45.690244 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:45.689982    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:45.690244 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:45.690024    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:45.912743 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:45.912700    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="bf4f35a7be19c935bd60022aede459693f7020c27e972b8207aeb0ba56b7ec0e" exitCode=0
Jul 28 19:35:45.912907 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:45.912792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"bf4f35a7be19c935bd60022aede459693f7020c27e972b8207aeb0ba56b7ec0e"}
Jul 28 19:35:47.689650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:47.689623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:47.689650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:47.689654    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:47.690190 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:47.689715    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:47.690190 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:47.689746    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:47.690190 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:47.689799    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:47.690190 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:47.689880    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:47.919821 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:47.919785    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="b4d81ea70a961b9af769d0611db9bf58659646c9734886e55679fa7a02bd7b3e" exitCode=0
Jul 28 19:35:47.919821 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:47.919824    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"b4d81ea70a961b9af769d0611db9bf58659646c9734886e55679fa7a02bd7b3e"}
Jul 28 19:35:49.240140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.240105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:49.240581 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.240151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:49.240581 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.240260    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:49.240581 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.240319    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs podName:777f220c-e1cb-4826-a60f-5546aabb3a23 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:21.240305779 +0000 UTC m=+65.129182728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs") pod "network-metrics-daemon-vmhbw" (UID: "777f220c-e1cb-4826-a60f-5546aabb3a23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:49.240581 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.240260    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:49.240581 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.240351    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret podName:b7d86f9a-aae2-46d4-b6d3-6831fea265bc nodeName:}" failed. No retries permitted until 2026-07-28 19:36:21.240343527 +0000 UTC m=+65.129220475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret") pod "global-pull-secret-syncer-cgvwx" (UID: "b7d86f9a-aae2-46d4-b6d3-6831fea265bc") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:49.341409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.341380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:49.341573 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.341535    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:49.341573 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.341552    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:49.341573 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.341563    2575 projected.go:196] Error preparing data for projected volume kube-api-access-xf4xs for pod openshift-network-diagnostics/network-check-target-vr28t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:49.341753 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.341616    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs podName:6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:21.341601209 +0000 UTC m=+65.230478160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xf4xs" (UniqueName: "kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs") pod "network-check-target-vr28t" (UID: "6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:49.689132 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.689097    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:49.689410 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.689150    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:49.689410 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:49.689150    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:49.689410 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.689230    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:49.689410 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.689361    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:49.689644 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:49.689451    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:51.689910 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:51.689875    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:51.690341 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:51.689915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:51.690341 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:51.690003    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:51.690341 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:51.690005    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:51.690341 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:51.690084    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:51.690341 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:51.690195    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:51.931540 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:51.931513    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="9d344ce6f92d5dbf58060f7b4d767a5d51ab43d1b4e05a23843b249fd93e37ed" exitCode=0
Jul 28 19:35:51.931661 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:51.931582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"9d344ce6f92d5dbf58060f7b4d767a5d51ab43d1b4e05a23843b249fd93e37ed"}
Jul 28 19:35:52.936391 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:52.936356    2575 generic.go:356] "Generic (PLEG): container finished" podID="877d18f7-d586-468e-9c8b-03870444164e" containerID="a94636c73d75f1540ff1fe1b01b8ed6fa157324ccbc51eb48e6a96672b04b430" exitCode=0
Jul 28 19:35:52.936743 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:52.936404    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerDied","Data":"a94636c73d75f1540ff1fe1b01b8ed6fa157324ccbc51eb48e6a96672b04b430"}
Jul 28 19:35:53.689010 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:53.688982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:53.689010 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:53.688982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:53.689204 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:53.689013    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:53.689204 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:53.689081    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:53.689204 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:53.689134    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:53.689204 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:53.689197    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:53.942433 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:53.942354    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" event={"ID":"877d18f7-d586-468e-9c8b-03870444164e","Type":"ContainerStarted","Data":"4baa1c50e23c5ec65455314f24dc9ec38c6b1f1dc668bb3c17109419e85d8c5c"}
Jul 28 19:35:53.971165 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:53.971120    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q8zqz" podStartSLOduration=8.352602037 podStartE2EDuration="37.971104599s" totalImagesPullingTime="29.618502562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:53.970570002 +0000 UTC m=+37.859446973" watchObservedRunningTime="2026-07-28 19:35:53.971104599 +0000 UTC m=+37.859981548"
Jul 28 19:35:55.689779 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:55.689747    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:55.689779 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:55.689772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:55.690225 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:55.689848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:55.690225 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:55.689852    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:55.690225 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:55.689915    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:55.690225 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:55.690012    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:57.689593 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:57.689558    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:57.689962 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:57.689569    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:57.689962 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:57.689674    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:35:57.689962 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:57.689575    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:57.689962 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:57.689722    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:57.689962 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:57.689830    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:59.689508 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:59.689475    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:35:59.689508 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:59.689501    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:35:59.689957 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:35:59.689477    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:35:59.689957 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:59.689575    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:35:59.689957 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:59.689643    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:35:59.689957 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:35:59.689718    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:01.689683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:01.689504    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:01.690228 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:01.689555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:01.690228 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:01.689779    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:36:01.690228 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:01.689565    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:01.690228 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:01.689837    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:01.690228 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:01.689893    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:36:03.688914 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.688880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:03.689492 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.688920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:03.689492 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.688922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:03.689492 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.688993    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:36:03.689492 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.689088    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:03.689492 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.689152    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:36:03.810635 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.810593    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cgvwx"]
Jul 28 19:36:03.811018 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.810951    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vr28t"]
Jul 28 19:36:03.813363 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.813316    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmhbw"]
Jul 28 19:36:03.964439 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.964408    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:03.964606 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.964412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:03.964606 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.964527    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:03.964606 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:03.964412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:03.964606 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.964573    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:36:03.964853 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:03.964634    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:36:05.689896 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:05.689865    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:05.689896 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:05.689884    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:05.689896 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:05.689900    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:05.690416 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:05.689985    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:36:05.690416 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:05.690113    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:05.690416 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:05.690219    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:36:07.689368 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.689330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:07.689948 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.689342    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:07.689948 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:07.689435    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmhbw" podUID="777f220c-e1cb-4826-a60f-5546aabb3a23"
Jul 28 19:36:07.689948 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:07.689525    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vr28t" podUID="6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1"
Jul 28 19:36:07.689948 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.689342    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:07.689948 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:07.689622    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgvwx" podUID="b7d86f9a-aae2-46d4-b6d3-6831fea265bc"
Jul 28 19:36:07.946095 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.946032    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-70.ec2.internal" event="NodeReady"
Jul 28 19:36:07.946220 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.946143    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:36:07.977923 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.977897    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8647557848-sq5kj"]
Jul 28 19:36:07.999897 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.999874    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-czt8t"]
Jul 28 19:36:08.000016 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:07.999996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.002218 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.002196    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p5kr\""
Jul 28 19:36:08.002339 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.002220    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:36:08.002339 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.002240    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:36:08.002339 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.002291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:36:08.013301 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.013279    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:36:08.015979 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.015960    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8647557848-sq5kj"]
Jul 28 19:36:08.016078 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.015990    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfpc"]
Jul 28 19:36:08.016136 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.016077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.017968 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.017941    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:36:08.018226 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.018209    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:36:08.018310 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.018297    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lt6xm\""
Jul 28 19:36:08.037852 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.037835    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czt8t"]
Jul 28 19:36:08.037950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.037864    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfpc"]
Jul 28 19:36:08.037998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.037950    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.039997 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.039982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:36:08.040112 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.040094    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:36:08.040222 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.040208    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:36:08.040297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.040283    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rtz66\""
Jul 28 19:36:08.040343 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.040330    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:36:08.084551 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-certificates\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084667 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96ef683-1e6b-4900-9520-84e87a7fe170-tmp-dir\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.084667 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-crio-socket\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.084667 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c96ef683-1e6b-4900-9520-84e87a7fe170-metrics-tls\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.084667 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084637    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084681    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2786\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-kube-api-access-g2786\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084725    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-ca-trust-extracted\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-data-volume\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrb6j\" (UniqueName: \"kubernetes.io/projected/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-api-access-jrb6j\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-image-registry-private-configuration\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9f5\" (UniqueName: \"kubernetes.io/projected/c96ef683-1e6b-4900-9520-84e87a7fe170-kube-api-access-ln9f5\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-tls\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-trusted-ca\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.084950 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-installation-pull-secrets\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.085236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-bound-sa-token\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.085236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.084994    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c96ef683-1e6b-4900-9520-84e87a7fe170-config-volume\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.093652 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.093628    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2rblv"]
Jul 28 19:36:08.116095 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.116072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.116188 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.116071    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2rblv"]
Jul 28 19:36:08.118284 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.118266    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:36:08.118388 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.118302    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:36:08.118388 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.118364    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpl6j\""
Jul 28 19:36:08.118510 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.118387    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:36:08.185874 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185841    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96ef683-1e6b-4900-9520-84e87a7fe170-tmp-dir\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185879    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-crio-socket\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c96ef683-1e6b-4900-9520-84e87a7fe170-metrics-tls\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185927    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2786\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-kube-api-access-g2786\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.185979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-ca-trust-extracted\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txmc4\" (UniqueName: \"kubernetes.io/projected/5369d78b-34e1-45f1-ac78-5260c5f7027b-kube-api-access-txmc4\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186062    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-data-volume\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrb6j\" (UniqueName: \"kubernetes.io/projected/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-api-access-jrb6j\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186095    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-crio-socket\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-image-registry-private-configuration\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5369d78b-34e1-45f1-ac78-5260c5f7027b-cert\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9f5\" (UniqueName: \"kubernetes.io/projected/c96ef683-1e6b-4900-9520-84e87a7fe170-kube-api-access-ln9f5\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186227    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-tls\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-trusted-ca\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-installation-pull-secrets\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186390 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96ef683-1e6b-4900-9520-84e87a7fe170-tmp-dir\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-ca-trust-extracted\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-data-volume\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-bound-sa-token\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c96ef683-1e6b-4900-9520-84e87a7fe170-config-volume\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.186904 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.186688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-certificates\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.187803 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.187595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-certificates\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.187803 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.187752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c96ef683-1e6b-4900-9520-84e87a7fe170-config-volume\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.188169 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.188015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-trusted-ca\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.190655 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.190631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c96ef683-1e6b-4900-9520-84e87a7fe170-metrics-tls\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.190778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.190691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-image-registry-private-configuration\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.190778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.190714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-installation-pull-secrets\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.190778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.190721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.190778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.190763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-registry-tls\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.194299 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.194266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2786\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-kube-api-access-g2786\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.194399 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.194357    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrb6j\" (UniqueName: \"kubernetes.io/projected/72b1deed-76bb-4fa0-bacb-e4c8e915fabb-kube-api-access-jrb6j\") pod \"insights-runtime-extractor-ssfpc\" (UID: \"72b1deed-76bb-4fa0-bacb-e4c8e915fabb\") " pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.194466 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.194414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e904d8f-e0f2-46c6-99fe-db4c5a883f59-bound-sa-token\") pod \"image-registry-8647557848-sq5kj\" (UID: \"6e904d8f-e0f2-46c6-99fe-db4c5a883f59\") " pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.194827 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.194813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9f5\" (UniqueName: \"kubernetes.io/projected/c96ef683-1e6b-4900-9520-84e87a7fe170-kube-api-access-ln9f5\") pod \"dns-default-czt8t\" (UID: \"c96ef683-1e6b-4900-9520-84e87a7fe170\") " pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.288128 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.288041    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txmc4\" (UniqueName: \"kubernetes.io/projected/5369d78b-34e1-45f1-ac78-5260c5f7027b-kube-api-access-txmc4\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.288128 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.288071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5369d78b-34e1-45f1-ac78-5260c5f7027b-cert\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.290166 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.290148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5369d78b-34e1-45f1-ac78-5260c5f7027b-cert\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.295681 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.295655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txmc4\" (UniqueName: \"kubernetes.io/projected/5369d78b-34e1-45f1-ac78-5260c5f7027b-kube-api-access-txmc4\") pod \"ingress-canary-2rblv\" (UID: \"5369d78b-34e1-45f1-ac78-5260c5f7027b\") " pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.313399 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.313379    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.324513 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.324482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:08.346516 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.346499    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssfpc"
Jul 28 19:36:08.423813 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.423782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2rblv"
Jul 28 19:36:08.555108 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.555080    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfpc"]
Jul 28 19:36:08.558684 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.558658    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8647557848-sq5kj"]
Jul 28 19:36:08.560291 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:08.559211    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b1deed_76bb_4fa0_bacb_e4c8e915fabb.slice/crio-4e1bb51e78f83e67164b82ee803dfa7083f817700de7d455fd74f8acff2c8ea5 WatchSource:0}: Error finding container 4e1bb51e78f83e67164b82ee803dfa7083f817700de7d455fd74f8acff2c8ea5: Status 404 returned error can't find the container with id 4e1bb51e78f83e67164b82ee803dfa7083f817700de7d455fd74f8acff2c8ea5
Jul 28 19:36:08.562067 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:08.562045    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e904d8f_e0f2_46c6_99fe_db4c5a883f59.slice/crio-10fd943910d5e0771a663fcf11b60011b4b3d69614aee56d7982381690bcfbd1 WatchSource:0}: Error finding container 10fd943910d5e0771a663fcf11b60011b4b3d69614aee56d7982381690bcfbd1: Status 404 returned error can't find the container with id 10fd943910d5e0771a663fcf11b60011b4b3d69614aee56d7982381690bcfbd1
Jul 28 19:36:08.564510 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.564488    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czt8t"]
Jul 28 19:36:08.568156 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:08.568134    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96ef683_1e6b_4900_9520_84e87a7fe170.slice/crio-b53e2ea4f97dea672a76bc6b09b2f5d1e3c9c3c4f080e72add95f9a006284a30 WatchSource:0}: Error finding container b53e2ea4f97dea672a76bc6b09b2f5d1e3c9c3c4f080e72add95f9a006284a30: Status 404 returned error can't find the container with id b53e2ea4f97dea672a76bc6b09b2f5d1e3c9c3c4f080e72add95f9a006284a30
Jul 28 19:36:08.595470 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.595434    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2rblv"]
Jul 28 19:36:08.599294 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:08.599260    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5369d78b_34e1_45f1_ac78_5260c5f7027b.slice/crio-b8c7ab7e7737ce0c21cfc0b51e0bf4479b950a7daf07aadbebd6dbd18d98da1e WatchSource:0}: Error finding container b8c7ab7e7737ce0c21cfc0b51e0bf4479b950a7daf07aadbebd6dbd18d98da1e: Status 404 returned error can't find the container with id b8c7ab7e7737ce0c21cfc0b51e0bf4479b950a7daf07aadbebd6dbd18d98da1e
Jul 28 19:36:08.976489 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.976243    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8647557848-sq5kj" event={"ID":"6e904d8f-e0f2-46c6-99fe-db4c5a883f59","Type":"ContainerStarted","Data":"a38e86e1c105e5096b8f5c665fe3a5d87f8f25871dad9e0a4dad6c619a11db80"}
Jul 28 19:36:08.976949 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.976523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8647557848-sq5kj" event={"ID":"6e904d8f-e0f2-46c6-99fe-db4c5a883f59","Type":"ContainerStarted","Data":"10fd943910d5e0771a663fcf11b60011b4b3d69614aee56d7982381690bcfbd1"}
Jul 28 19:36:08.976949 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.976558    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:08.981574 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.981542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2rblv" event={"ID":"5369d78b-34e1-45f1-ac78-5260c5f7027b","Type":"ContainerStarted","Data":"b8c7ab7e7737ce0c21cfc0b51e0bf4479b950a7daf07aadbebd6dbd18d98da1e"}
Jul 28 19:36:08.982844 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.982817    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfpc" event={"ID":"72b1deed-76bb-4fa0-bacb-e4c8e915fabb","Type":"ContainerStarted","Data":"4e1bb51e78f83e67164b82ee803dfa7083f817700de7d455fd74f8acff2c8ea5"}
Jul 28 19:36:08.984179 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.984096    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czt8t" event={"ID":"c96ef683-1e6b-4900-9520-84e87a7fe170","Type":"ContainerStarted","Data":"b53e2ea4f97dea672a76bc6b09b2f5d1e3c9c3c4f080e72add95f9a006284a30"}
Jul 28 19:36:08.995896 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:08.995669    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8647557848-sq5kj" podStartSLOduration=6.9956531680000005 podStartE2EDuration="6.995653168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:08.995329335 +0000 UTC m=+52.884206307" watchObservedRunningTime="2026-07-28 19:36:08.995653168 +0000 UTC m=+52.884530139"
Jul 28 19:36:09.690300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.689244    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:09.690300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.689243    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:09.690300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.689397    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.694435    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vbfhc\""
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.694817    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.695015    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.695201    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.695393    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:36:09.697227 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:09.695588    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jglff\""
Jul 28 19:36:14.925960 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:14.925929    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9nxc"
Jul 28 19:36:15.004603 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.004563    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfpc" event={"ID":"72b1deed-76bb-4fa0-bacb-e4c8e915fabb","Type":"ContainerStarted","Data":"5854d3bc9352483eef9fccb7cee56fd074b9244e500ccd3cd5c42c4fb0bc3a5a"}
Jul 28 19:36:15.006767 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.006721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czt8t" event={"ID":"c96ef683-1e6b-4900-9520-84e87a7fe170","Type":"ContainerStarted","Data":"a5f66dd0c427cb921b6eb0a1dd8729d40d10b1c41ec2a323281a9784d3b3ea46"}
Jul 28 19:36:15.006901 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.006792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czt8t" event={"ID":"c96ef683-1e6b-4900-9520-84e87a7fe170","Type":"ContainerStarted","Data":"386b7a8b205643d3e03fafbccaaa643a86a59977e020774b6b9cb651e8942fa6"}
Jul 28 19:36:15.006901 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.006824    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:15.008801 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.008776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2rblv" event={"ID":"5369d78b-34e1-45f1-ac78-5260c5f7027b","Type":"ContainerStarted","Data":"bb0ed20cf2d928b43dcef7b80cd05ff7ab78d3211d67d20e66915ec0e9ef752b"}
Jul 28 19:36:15.024594 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.024544    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-czt8t" podStartSLOduration=2.491401806 podStartE2EDuration="8.024525408s" totalImagesPullingTime="5.533123602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:15.023946878 +0000 UTC m=+58.912823878" watchObservedRunningTime="2026-07-28 19:36:15.024525408 +0000 UTC m=+58.913402359"
Jul 28 19:36:15.039879 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:15.039830    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2rblv" podStartSLOduration=1.539387274 podStartE2EDuration="7.039810775s" totalImagesPullingTime="5.500423501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:15.038819804 +0000 UTC m=+58.927696773" watchObservedRunningTime="2026-07-28 19:36:15.039810775 +0000 UTC m=+58.928687751"
Jul 28 19:36:18.021075 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:18.021040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfpc" event={"ID":"72b1deed-76bb-4fa0-bacb-e4c8e915fabb","Type":"ContainerStarted","Data":"5e5c77b96651f283476eff39ef77f5e739bac5edcc0f70679bf2d2291a300ca0"}
Jul 28 19:36:21.307516 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.307480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:21.307516 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.307530    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:21.309864 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.309842    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:36:21.309989 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.309884    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:36:21.322060 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.322032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/777f220c-e1cb-4826-a60f-5546aabb3a23-metrics-certs\") pod \"network-metrics-daemon-vmhbw\" (UID: \"777f220c-e1cb-4826-a60f-5546aabb3a23\") " pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:21.322153 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.322066    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d86f9a-aae2-46d4-b6d3-6831fea265bc-original-pull-secret\") pod \"global-pull-secret-syncer-cgvwx\" (UID: \"b7d86f9a-aae2-46d4-b6d3-6831fea265bc\") " pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:21.408534 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.408499    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:21.410957 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.410935    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:36:21.418894 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.418871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgvwx"
Jul 28 19:36:21.420584 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.420565    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:36:21.430431 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.430404    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vbfhc\""
Jul 28 19:36:21.432247 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.432227    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf4xs\" (UniqueName: \"kubernetes.io/projected/6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1-kube-api-access-xf4xs\") pod \"network-check-target-vr28t\" (UID: \"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1\") " pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:21.439296 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.439278    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmhbw"
Jul 28 19:36:21.550982 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.550950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cgvwx"]
Jul 28 19:36:21.553786 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:21.553760    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d86f9a_aae2_46d4_b6d3_6831fea265bc.slice/crio-28b2b6facfe5e6ee6403192df0fd840b835859908e2fdd2c5616532711eed4dd WatchSource:0}: Error finding container 28b2b6facfe5e6ee6403192df0fd840b835859908e2fdd2c5616532711eed4dd: Status 404 returned error can't find the container with id 28b2b6facfe5e6ee6403192df0fd840b835859908e2fdd2c5616532711eed4dd
Jul 28 19:36:21.566272 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.566249    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmhbw"]
Jul 28 19:36:21.568962 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:21.568941    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777f220c_e1cb_4826_a60f_5546aabb3a23.slice/crio-e403b412b6aba6fdacfff84bc4f2fdcdcd7becbafdeb274577349a29dd793cc0 WatchSource:0}: Error finding container e403b412b6aba6fdacfff84bc4f2fdcdcd7becbafdeb274577349a29dd793cc0: Status 404 returned error can't find the container with id e403b412b6aba6fdacfff84bc4f2fdcdcd7becbafdeb274577349a29dd793cc0
Jul 28 19:36:21.710670 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.710596    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jglff\""
Jul 28 19:36:21.719396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.719376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:21.867679 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:21.867648    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vr28t"]
Jul 28 19:36:21.870101 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:21.870077    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8fe7ed_3845_42c9_b3d4_be8b1a3ef8a1.slice/crio-5559d48b54025178ffd6ecc30150374aa82bc332901525495eeb9e15eca9aa12 WatchSource:0}: Error finding container 5559d48b54025178ffd6ecc30150374aa82bc332901525495eeb9e15eca9aa12: Status 404 returned error can't find the container with id 5559d48b54025178ffd6ecc30150374aa82bc332901525495eeb9e15eca9aa12
Jul 28 19:36:22.037101 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:22.037023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmhbw" event={"ID":"777f220c-e1cb-4826-a60f-5546aabb3a23","Type":"ContainerStarted","Data":"e403b412b6aba6fdacfff84bc4f2fdcdcd7becbafdeb274577349a29dd793cc0"}
Jul 28 19:36:22.038308 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:22.038278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vr28t" event={"ID":"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1","Type":"ContainerStarted","Data":"5559d48b54025178ffd6ecc30150374aa82bc332901525495eeb9e15eca9aa12"}
Jul 28 19:36:22.039561 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:22.039535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cgvwx" event={"ID":"b7d86f9a-aae2-46d4-b6d3-6831fea265bc","Type":"ContainerStarted","Data":"28b2b6facfe5e6ee6403192df0fd840b835859908e2fdd2c5616532711eed4dd"}
Jul 28 19:36:22.041678 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:22.041654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfpc" event={"ID":"72b1deed-76bb-4fa0-bacb-e4c8e915fabb","Type":"ContainerStarted","Data":"43cfbea4a843f7a89e42c8acb863875cdab2fc3821ce9786c98717e8233ad9f8"}
Jul 28 19:36:22.060984 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:22.060943    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ssfpc" podStartSLOduration=2.560314452 podStartE2EDuration="15.06092912s" totalImagesPullingTime="12.500614668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:22.060352927 +0000 UTC m=+65.949229904" watchObservedRunningTime="2026-07-28 19:36:22.06092912 +0000 UTC m=+65.949806087"
Jul 28 19:36:23.049477 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:23.049436    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmhbw" event={"ID":"777f220c-e1cb-4826-a60f-5546aabb3a23","Type":"ContainerStarted","Data":"d77a8fa1d2ec60b461abb74b285ba76d38a24977ceb1260eba5c70e51ce8eaf0"}
Jul 28 19:36:24.055520 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:24.055475    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmhbw" event={"ID":"777f220c-e1cb-4826-a60f-5546aabb3a23","Type":"ContainerStarted","Data":"fa76ce29e163e4361307e3b492b6f329a4685c71679d2d863517e6e59cccbb4b"}
Jul 28 19:36:24.075098 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:24.075048    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vmhbw" podStartSLOduration=66.746466032 podStartE2EDuration="1m8.075029091s" totalImagesPullingTime="1.328563059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:24.073916379 +0000 UTC m=+67.962793350" watchObservedRunningTime="2026-07-28 19:36:24.075029091 +0000 UTC m=+67.963906063"
Jul 28 19:36:25.015589 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:25.015552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-czt8t"
Jul 28 19:36:26.218484 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.218451    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"]
Jul 28 19:36:26.221591 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.221568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:26.223667 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.223642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 19:36:26.223878 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.223855    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gzzbf\""
Jul 28 19:36:26.231097 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.231076    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"]
Jul 28 19:36:26.347233 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.347193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54e6bb1-2403-4917-9e45-822bcf490511-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4whjh\" (UID: \"e54e6bb1-2403-4917-9e45-822bcf490511\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:26.448652 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.448617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54e6bb1-2403-4917-9e45-822bcf490511-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4whjh\" (UID: \"e54e6bb1-2403-4917-9e45-822bcf490511\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:26.451342 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.451310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e54e6bb1-2403-4917-9e45-822bcf490511-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-4whjh\" (UID: \"e54e6bb1-2403-4917-9e45-822bcf490511\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:26.532785 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.532657    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:26.781233 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:26.781206    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"]
Jul 28 19:36:26.784580 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:26.784558    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54e6bb1_2403_4917_9e45_822bcf490511.slice/crio-576a5f6fa355f4ea93b0b77797ee2cac5b96c6293df0fb4966d9bbbed26582c4 WatchSource:0}: Error finding container 576a5f6fa355f4ea93b0b77797ee2cac5b96c6293df0fb4966d9bbbed26582c4: Status 404 returned error can't find the container with id 576a5f6fa355f4ea93b0b77797ee2cac5b96c6293df0fb4966d9bbbed26582c4
Jul 28 19:36:27.066771 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.066658    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vr28t" event={"ID":"6b8fe7ed-3845-42c9-b3d4-be8b1a3ef8a1","Type":"ContainerStarted","Data":"6e78aec3a6136c4ad07e33e190df8fe530e353103e53ecb98e781d0a7fd068b8"}
Jul 28 19:36:27.066771 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.066765    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:36:27.068213 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.068187    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cgvwx" event={"ID":"b7d86f9a-aae2-46d4-b6d3-6831fea265bc","Type":"ContainerStarted","Data":"6ef9e5351377e1748a0c2354bf836d8088fd5a4e7ef06d93ce0ce98bbaecb55b"}
Jul 28 19:36:27.069281 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.069263    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh" event={"ID":"e54e6bb1-2403-4917-9e45-822bcf490511","Type":"ContainerStarted","Data":"576a5f6fa355f4ea93b0b77797ee2cac5b96c6293df0fb4966d9bbbed26582c4"}
Jul 28 19:36:27.081633 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.081590    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vr28t" podStartSLOduration=66.296112331 podStartE2EDuration="1m11.081576727s" totalImagesPullingTime="4.785464396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:27.081074451 +0000 UTC m=+70.969951424" watchObservedRunningTime="2026-07-28 19:36:27.081576727 +0000 UTC m=+70.970453697"
Jul 28 19:36:27.095288 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:27.095250    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cgvwx" podStartSLOduration=65.987635456 podStartE2EDuration="1m11.09523716s" totalImagesPullingTime="5.107601704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:27.094436187 +0000 UTC m=+70.983313158" watchObservedRunningTime="2026-07-28 19:36:27.09523716 +0000 UTC m=+70.984114131"
Jul 28 19:36:28.074517 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:28.074477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh" event={"ID":"e54e6bb1-2403-4917-9e45-822bcf490511","Type":"ContainerStarted","Data":"3d7c960732ff930b327155979a81247b4a86048cbbfd61f9b9d7773cc0e28f3a"}
Jul 28 19:36:28.089948 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:28.089893    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh" podStartSLOduration=0.937994394 podStartE2EDuration="2.089877528s" totalImagesPullingTime="1.151883134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:28.089487115 +0000 UTC m=+71.978364090" watchObservedRunningTime="2026-07-28 19:36:28.089877528 +0000 UTC m=+71.978754498"
Jul 28 19:36:29.077682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:29.077654    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:29.082428 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:29.082408    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-4whjh"
Jul 28 19:36:29.992578 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:29.992553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8647557848-sq5kj"
Jul 28 19:36:33.615162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.615127    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"]
Jul 28 19:36:33.618282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.618266    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.621409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621370    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 19:36:33.621409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621393    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nzsqx\""
Jul 28 19:36:33.621577 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621381    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:36:33.621577 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621381    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:36:33.621577 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621428    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:36:33.621577 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.621381    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:36:33.628405 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.628385    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"]
Jul 28 19:36:33.650717 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.650694    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"]
Jul 28 19:36:33.653568 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.653553    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.656261 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.656244    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:36:33.656461 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.656444    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bxcdg\""
Jul 28 19:36:33.656515 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.656446    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 19:36:33.656571 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.656555    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 19:36:33.676586 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.676560    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"]
Jul 28 19:36:33.683021 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.683001    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rxh9w"]
Jul 28 19:36:33.685909 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.685893    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.688281 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.688265    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8dtwl\""
Jul 28 19:36:33.688358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.688321    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:36:33.688585 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.688569    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:36:33.688638 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.688577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:36:33.695361 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.695341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.695438 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.695373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.695438 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.695397    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.695438 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.695417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drss\" (UniqueName: \"kubernetes.io/projected/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-kube-api-access-8drss\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.795954 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.795915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drss\" (UniqueName: \"kubernetes.io/projected/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-kube-api-access-8drss\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.796126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.795959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-root\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.795990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.796126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f86w6\" (UniqueName: \"kubernetes.io/projected/a7c4da5c-8150-4a47-a48e-28cc564d77a3-kube-api-access-f86w6\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796219    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-wtmp\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796317    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2d88fce-2d42-4c48-81ac-b615150d64fb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796401    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vxc\" (UniqueName: \"kubernetes.io/projected/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-api-access-74vxc\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-metrics-client-ca\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-sys\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-textfile\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.796683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.796998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.796861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.798596 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.798568    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.798596 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.798588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.804105 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.804085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drss\" (UniqueName: \"kubernetes.io/projected/8ff38f0a-2b30-4f78-baf3-c97620f1a92b-kube-api-access-8drss\") pod \"openshift-state-metrics-b8f49f5c6-fwqnv\" (UID: \"8ff38f0a-2b30-4f78-baf3-c97620f1a92b\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.897971 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.897893    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-textfile\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.897971 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.897933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.897971 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.897961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.897991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-root\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f86w6\" (UniqueName: \"kubernetes.io/projected/a7c4da5c-8150-4a47-a48e-28cc564d77a3-kube-api-access-f86w6\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-wtmp\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-root\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898268    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-textfile\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2d88fce-2d42-4c48-81ac-b615150d64fb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vxc\" (UniqueName: \"kubernetes.io/projected/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-api-access-74vxc\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-metrics-client-ca\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-sys\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:33.898357    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:36:33.898507 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:36:33.898515    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls podName:a7c4da5c-8150-4a47-a48e-28cc564d77a3 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:34.398495391 +0000 UTC m=+78.287372340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls") pod "node-exporter-rxh9w" (UID: "a7c4da5c-8150-4a47-a48e-28cc564d77a3") : secret "node-exporter-tls" not found
Jul 28 19:36:33.899014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898606    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e2d88fce-2d42-4c48-81ac-b615150d64fb-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.899014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-sys\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.899014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-wtmp\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.899014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.898751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.899251 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.899216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.899910 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.899884    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7c4da5c-8150-4a47-a48e-28cc564d77a3-metrics-client-ca\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.902126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.902104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.904529 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.904500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2d88fce-2d42-4c48-81ac-b615150d64fb-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.905363 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.905337    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.905473 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.905453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.906708 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.906685    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f86w6\" (UniqueName: \"kubernetes.io/projected/a7c4da5c-8150-4a47-a48e-28cc564d77a3-kube-api-access-f86w6\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:33.907147 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.907117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vxc\" (UniqueName: \"kubernetes.io/projected/e2d88fce-2d42-4c48-81ac-b615150d64fb-kube-api-access-74vxc\") pod \"kube-state-metrics-56d9f944b4-fzm8r\" (UID: \"e2d88fce-2d42-4c48-81ac-b615150d64fb\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:33.927135 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.927114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"
Jul 28 19:36:33.961969 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:33.961936    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"
Jul 28 19:36:34.057762 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.057699    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv"]
Jul 28 19:36:34.061041 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:34.061001    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff38f0a_2b30_4f78_baf3_c97620f1a92b.slice/crio-7c1876c4d75b64c22023b5241be84ef1b1a645168e1c197745fde23d83676ebf WatchSource:0}: Error finding container 7c1876c4d75b64c22023b5241be84ef1b1a645168e1c197745fde23d83676ebf: Status 404 returned error can't find the container with id 7c1876c4d75b64c22023b5241be84ef1b1a645168e1c197745fde23d83676ebf
Jul 28 19:36:34.093952 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.093891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv" event={"ID":"8ff38f0a-2b30-4f78-baf3-c97620f1a92b","Type":"ContainerStarted","Data":"7c1876c4d75b64c22023b5241be84ef1b1a645168e1c197745fde23d83676ebf"}
Jul 28 19:36:34.108634 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.108608    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r"]
Jul 28 19:36:34.110902 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:34.110876    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d88fce_2d42_4c48_81ac_b615150d64fb.slice/crio-3a4509a7b4ebb972ef09c19f1a7cb68fb51bb8843527b3825ae8e55c42a9d905 WatchSource:0}: Error finding container 3a4509a7b4ebb972ef09c19f1a7cb68fb51bb8843527b3825ae8e55c42a9d905: Status 404 returned error can't find the container with id 3a4509a7b4ebb972ef09c19f1a7cb68fb51bb8843527b3825ae8e55c42a9d905
Jul 28 19:36:34.402083 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.402000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:34.404223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.404192    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7c4da5c-8150-4a47-a48e-28cc564d77a3-node-exporter-tls\") pod \"node-exporter-rxh9w\" (UID: \"a7c4da5c-8150-4a47-a48e-28cc564d77a3\") " pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:34.594239 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:34.594202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxh9w"
Jul 28 19:36:34.606474 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:34.606430    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c4da5c_8150_4a47_a48e_28cc564d77a3.slice/crio-e56b6c3f4cc7a875006186727e9a07c058c9080998ee729b81edc7a312fa6345 WatchSource:0}: Error finding container e56b6c3f4cc7a875006186727e9a07c058c9080998ee729b81edc7a312fa6345: Status 404 returned error can't find the container with id e56b6c3f4cc7a875006186727e9a07c058c9080998ee729b81edc7a312fa6345
Jul 28 19:36:35.100279 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.100237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv" event={"ID":"8ff38f0a-2b30-4f78-baf3-c97620f1a92b","Type":"ContainerStarted","Data":"a23fe121871f9195f4abd23dc2197fdd525884016ee285b203b7e344efe5521d"}
Jul 28 19:36:35.100795 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.100314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv" event={"ID":"8ff38f0a-2b30-4f78-baf3-c97620f1a92b","Type":"ContainerStarted","Data":"985134f1b79c94983706b78c735b2abc157bb84395c3c52f555c8b1813d3ffaa"}
Jul 28 19:36:35.101682 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.101651    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxh9w" event={"ID":"a7c4da5c-8150-4a47-a48e-28cc564d77a3","Type":"ContainerStarted","Data":"e56b6c3f4cc7a875006186727e9a07c058c9080998ee729b81edc7a312fa6345"}
Jul 28 19:36:35.103195 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.103172    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r" event={"ID":"e2d88fce-2d42-4c48-81ac-b615150d64fb","Type":"ContainerStarted","Data":"3a4509a7b4ebb972ef09c19f1a7cb68fb51bb8843527b3825ae8e55c42a9d905"}
Jul 28 19:36:35.672628 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.672597    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7574b74685-zl65p"]
Jul 28 19:36:35.678370 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.677782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.680404 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.680380    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 19:36:35.680881 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.680862    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6xbz2\""
Jul 28 19:36:35.681001 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.680984    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 19:36:35.681145 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.681131    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 19:36:35.681223 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.681203    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-13ibnoehb687r\""
Jul 28 19:36:35.681280 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.681260    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 19:36:35.681423 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.680904    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 19:36:35.692850 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.692768    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7574b74685-zl65p"]
Jul 28 19:36:35.813931 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.813857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cltjk\" (UniqueName: \"kubernetes.io/projected/c2e31e3c-9995-4079-b634-b0dca2866336-kube-api-access-cltjk\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.813931 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.813909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.813951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.813986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.814026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-grpc-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.814068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.814094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2e31e3c-9995-4079-b634-b0dca2866336-metrics-client-ca\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.814144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.814121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915241 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cltjk\" (UniqueName: \"kubernetes.io/projected/c2e31e3c-9995-4079-b634-b0dca2866336-kube-api-access-cltjk\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915241 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-grpc-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2e31e3c-9995-4079-b634-b0dca2866336-metrics-client-ca\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.915485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.915447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.916709 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.916651    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2e31e3c-9995-4079-b634-b0dca2866336-metrics-client-ca\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.918816 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.918762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.919475 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.919065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-grpc-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.919475 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.919426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-tls\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.919475 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.919462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.920006 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.919987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.920078 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.920057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2e31e3c-9995-4079-b634-b0dca2866336-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:35.931911 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:35.931860    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cltjk\" (UniqueName: \"kubernetes.io/projected/c2e31e3c-9995-4079-b634-b0dca2866336-kube-api-access-cltjk\") pod \"thanos-querier-7574b74685-zl65p\" (UID: \"c2e31e3c-9995-4079-b634-b0dca2866336\") " pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:36.002428 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.002389    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:36.115234 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.115201    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv" event={"ID":"8ff38f0a-2b30-4f78-baf3-c97620f1a92b","Type":"ContainerStarted","Data":"1d1aba9316fa113e8ce4427403e4fbce49c1ac01f77d8559f0040cbea936398a"}
Jul 28 19:36:36.117839 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.117786    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r" event={"ID":"e2d88fce-2d42-4c48-81ac-b615150d64fb","Type":"ContainerStarted","Data":"a7073470e58dd8b9b6253226fe38bc33c7f8d15761a69ce05e725932740e4b72"}
Jul 28 19:36:36.117915 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.117844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r" event={"ID":"e2d88fce-2d42-4c48-81ac-b615150d64fb","Type":"ContainerStarted","Data":"4e16a3f03626859f09c951fc518da4ef698a268c89274a535ffbd9badd32e714"}
Jul 28 19:36:36.117915 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.117879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r" event={"ID":"e2d88fce-2d42-4c48-81ac-b615150d64fb","Type":"ContainerStarted","Data":"ef9ec8c11fb2de5319f65cf4d56162809245fef2134f25b81afc443de52350c6"}
Jul 28 19:36:36.139061 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.138887    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-fwqnv" podStartSLOduration=1.8409780150000001 podStartE2EDuration="3.13886668s" totalImagesPullingTime="1.297888665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:36.138177698 +0000 UTC m=+80.027054669" watchObservedRunningTime="2026-07-28 19:36:36.13886668 +0000 UTC m=+80.027743652"
Jul 28 19:36:36.150279 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.150252    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7574b74685-zl65p"]
Jul 28 19:36:36.152930 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:36.152891    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e31e3c_9995_4079_b634_b0dca2866336.slice/crio-718b44b0c51cdedade4ac97f162e10b40a0f9126dd997c2eb771cf7e20f7f50c WatchSource:0}: Error finding container 718b44b0c51cdedade4ac97f162e10b40a0f9126dd997c2eb771cf7e20f7f50c: Status 404 returned error can't find the container with id 718b44b0c51cdedade4ac97f162e10b40a0f9126dd997c2eb771cf7e20f7f50c
Jul 28 19:36:36.167988 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:36.167936    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fzm8r" podStartSLOduration=1.813230235 podStartE2EDuration="3.167919527s" totalImagesPullingTime="1.354689292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:36.165423603 +0000 UTC m=+80.054300575" watchObservedRunningTime="2026-07-28 19:36:36.167919527 +0000 UTC m=+80.056796537"
Jul 28 19:36:37.123443 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:37.123403    2575 generic.go:356] "Generic (PLEG): container finished" podID="a7c4da5c-8150-4a47-a48e-28cc564d77a3" containerID="e82d4b838dc45cf032aacac29f8dd949786457dc4a3926fef1ffdd4c8c4ed4ee" exitCode=0
Jul 28 19:36:37.123908 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:37.123491    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxh9w" event={"ID":"a7c4da5c-8150-4a47-a48e-28cc564d77a3","Type":"ContainerDied","Data":"e82d4b838dc45cf032aacac29f8dd949786457dc4a3926fef1ffdd4c8c4ed4ee"}
Jul 28 19:36:37.125181 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:37.125092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"718b44b0c51cdedade4ac97f162e10b40a0f9126dd997c2eb771cf7e20f7f50c"}
Jul 28 19:36:38.130648 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:38.130613    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxh9w" event={"ID":"a7c4da5c-8150-4a47-a48e-28cc564d77a3","Type":"ContainerStarted","Data":"99b98d93a84cfa1d041e99e299ce7a76ddeaffab507a261a3d330fb5ead49d46"}
Jul 28 19:36:38.131027 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:38.130717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxh9w" event={"ID":"a7c4da5c-8150-4a47-a48e-28cc564d77a3","Type":"ContainerStarted","Data":"d829e6328cade6b92b916b8a382d7b6a7dadc41e99e8a8ccf3f4328b08bb553f"}
Jul 28 19:36:38.160527 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:38.160480    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rxh9w" podStartSLOduration=3.646936901 podStartE2EDuration="5.160464501s" totalImagesPullingTime="1.442535128s" totalInitContainerRuntime="70.992472ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:38.15861525 +0000 UTC m=+82.047492223" watchObservedRunningTime="2026-07-28 19:36:38.160464501 +0000 UTC m=+82.049341472"
Jul 28 19:36:39.137996 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.137959    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"8b68b6af126e2f596e8dfcb1c46ec48326040c3a6d54cc798d9aa9d02828c09d"}
Jul 28 19:36:39.138282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.138042    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"33cebfabc07f51507dad895b7d99dd585f8c2989f82e3ef27b381f402588acc8"}
Jul 28 19:36:39.138282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.138085    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"5317f1880f57e78375c37e8ed7038de3d821a84dba7a6ccc936c3d7a929f7cc7"}
Jul 28 19:36:39.138282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.138126    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"adefd5893577f1743d2ae7b5bced7510a613fa7319d683e4ec1ddac6b17984e0"}
Jul 28 19:36:39.138282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.138164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"ed59e28980f6e1fa5652631f6b5447748e102ff3a9750fc679e54af3395c5332"}
Jul 28 19:36:39.138282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.138191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" event={"ID":"c2e31e3c-9995-4079-b634-b0dca2866336","Type":"ContainerStarted","Data":"202721c743061ba488eb45582f06f6af4230b561294bae22c9e4377736876098"}
Jul 28 19:36:39.162155 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.162100    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p" podStartSLOduration=1.528533385 podStartE2EDuration="4.162084575s" totalImagesPullingTime="2.63355119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:39.160972286 +0000 UTC m=+83.049849255" watchObservedRunningTime="2026-07-28 19:36:39.162084575 +0000 UTC m=+83.050961545"
Jul 28 19:36:39.900298 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.900260    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:36:39.903806 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.903789    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:39.906592 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.906567    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 19:36:39.906864 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.906849    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 19:36:39.907164 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907146    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 19:36:39.907254 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 19:36:39.907254 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907171    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 19:36:39.907366 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907186    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 19:36:39.907366 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907195    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 19:36:39.907366 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.907308    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 19:36:39.908408 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.908391    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 19:36:39.908494 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.908438    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 19:36:39.908679 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.908665    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 19:36:39.910294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.909462    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 19:36:39.910294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.909514    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6tqk5\""
Jul 28 19:36:39.910294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.909812    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 19:36:39.910294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.910281    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4snbp9g6c9h02\""
Jul 28 19:36:39.920442 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.920417    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 19:36:39.922724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.922707    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 19:36:39.929959 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:39.929843    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:36:40.052881 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.052845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.052891    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.052916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.052953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.052981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-config-out\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-web-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053042 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053231 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053240    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.053429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.053414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4bph\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-kube-api-access-f4bph\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.142239 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.142209    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:40.154504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154496    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154678    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bph\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-kube-api-access-f4bph\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154780    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.154893 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-config-out\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-web-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.154976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.155005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.155269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.155660    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.155934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.155699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.156659 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.156632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.156912 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.156893    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.157472 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.157449    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.158269 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.157967    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80a50eb3-07fa-4875-a492-32f32387021e-config-out\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.158269 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.158023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.158610 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.158587    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.159232 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.158846    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.159232 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.159157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-web-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.159232 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.159199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/80a50eb3-07fa-4875-a492-32f32387021e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.159459 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.159320    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.159708 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.159573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-config\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.160060 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.160037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.160145 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.160044    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.160346 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.160328    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.160583 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.160562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.161409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.161388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/80a50eb3-07fa-4875-a492-32f32387021e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.165162 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.165133    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bph\" (UniqueName: \"kubernetes.io/projected/80a50eb3-07fa-4875-a492-32f32387021e-kube-api-access-f4bph\") pod \"prometheus-k8s-0\" (UID: \"80a50eb3-07fa-4875-a492-32f32387021e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.216186 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.216153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:40.350289 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:40.350258    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:36:40.353849 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:36:40.353812    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a50eb3_07fa_4875_a492_32f32387021e.slice/crio-e7d039eb0dc8bce1ea3b0101b2b703829ef1fc1139d32427869c08cfd1f5358b WatchSource:0}: Error finding container e7d039eb0dc8bce1ea3b0101b2b703829ef1fc1139d32427869c08cfd1f5358b: Status 404 returned error can't find the container with id e7d039eb0dc8bce1ea3b0101b2b703829ef1fc1139d32427869c08cfd1f5358b
Jul 28 19:36:41.148956 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:41.148909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"e7d039eb0dc8bce1ea3b0101b2b703829ef1fc1139d32427869c08cfd1f5358b"}
Jul 28 19:36:42.157653 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:42.157619    2575 generic.go:356] "Generic (PLEG): container finished" podID="80a50eb3-07fa-4875-a492-32f32387021e" containerID="e9104818df1dbf66f5cfa05978081c0747ad110f73da91286eed3265a0a26605" exitCode=0
Jul 28 19:36:42.158037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:42.157668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerDied","Data":"e9104818df1dbf66f5cfa05978081c0747ad110f73da91286eed3265a0a26605"}
Jul 28 19:36:46.154697 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.154674    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7574b74685-zl65p"
Jul 28 19:36:46.175937 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.175909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"b893b42e4365d053c1a25b15874ec4504d85d15d5d5c3d4f855248f26ee00ad6"}
Jul 28 19:36:46.176029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.175970    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"f46905d932f0c0cacf38dc361281e0d23d5358e9d882712ab103f51346e902d7"}
Jul 28 19:36:46.176029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.175997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"7bd8b194d099fe25dedfd7ecf341e59eb00c95ec7852f93054a096a43656f582"}
Jul 28 19:36:46.176029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.176019    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"cbb84d250c16c5a841957b88dc42628b10cc284ad03383cb1eb6c92ed2fd6fa3"}
Jul 28 19:36:46.176124 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:46.176041    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"ace946ac080e95072b84f27a2d419a7c4ad868e7b078711ff28e7806ec8c1bfe"}
Jul 28 19:36:47.182362 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:47.182325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"80a50eb3-07fa-4875-a492-32f32387021e","Type":"ContainerStarted","Data":"2a545c1a3f1c68cdfc1e4ac8e1bd0d926b35a4b9ccc7fb1c29a13b527e9dcb45"}
Jul 28 19:36:47.210837 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:47.210790    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.979405743 podStartE2EDuration="8.210772189s" totalImagesPullingTime="4.210561645s" totalInitContainerRuntime="20.804801ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:47.208871176 +0000 UTC m=+91.097748145" watchObservedRunningTime="2026-07-28 19:36:47.210772189 +0000 UTC m=+91.099649160"
Jul 28 19:36:50.216567 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:50.216535    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:36:58.077808 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:36:58.077767    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vr28t"
Jul 28 19:37:40.217085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:37:40.217040    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:37:40.240705 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:37:40.240672    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:37:40.393581 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:37:40.393544    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:40:16.488778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:16.488747    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:40:16.493396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:16.493374    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:40:16.504117 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:16.504091    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:40:23.404518 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.404479    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"]
Jul 28 19:40:23.406832 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.406815    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.409336 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.409307    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:40:23.409336 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.409318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k8nwg\""
Jul 28 19:40:23.409825 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.409808    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:40:23.409948 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.409930    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:40:23.410135 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.410119    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:40:23.422868 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.422848    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"]
Jul 28 19:40:23.508987 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.508949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.508987 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.508985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.509175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.509015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.509175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.509096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.509175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.509137    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzg9b\" (UniqueName: \"kubernetes.io/projected/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-kube-api-access-bzg9b\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.609971 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.609932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.611121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.610193    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.611121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.610272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.611121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.610304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.611121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.610353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzg9b\" (UniqueName: \"kubernetes.io/projected/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-kube-api-access-bzg9b\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.611121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.611028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.614172 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.614148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.614272 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.614193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.614407 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.614385    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.621189 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.621164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzg9b\" (UniqueName: \"kubernetes.io/projected/41dd91d3-1775-4fd6-bb7a-e746ab69fbd5-kube-api-access-bzg9b\") pod \"opendatahub-operator-controller-manager-5f4d69945-85b9t\" (UID: \"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.717155 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.717072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:23.854707 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.854553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"]
Jul 28 19:40:23.857486 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:40:23.857461    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41dd91d3_1775_4fd6_bb7a_e746ab69fbd5.slice/crio-371c40e629fab792c012e559bdc78d798405c66e84b48a2d4d97e9756ea6f2a4 WatchSource:0}: Error finding container 371c40e629fab792c012e559bdc78d798405c66e84b48a2d4d97e9756ea6f2a4: Status 404 returned error can't find the container with id 371c40e629fab792c012e559bdc78d798405c66e84b48a2d4d97e9756ea6f2a4
Jul 28 19:40:23.859377 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.859355    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:40:23.947901 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:23.947868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t" event={"ID":"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5","Type":"ContainerStarted","Data":"371c40e629fab792c012e559bdc78d798405c66e84b48a2d4d97e9756ea6f2a4"}
Jul 28 19:40:26.962498 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:26.962426    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t" event={"ID":"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5","Type":"ContainerStarted","Data":"7724ac99f4b871843adcaa2dcf75823d79c102cb44fba08a1937fa390678a681"}
Jul 28 19:40:27.967235 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:27.967195    2575 generic.go:356] "Generic (PLEG): container finished" podID="41dd91d3-1775-4fd6-bb7a-e746ab69fbd5" containerID="7724ac99f4b871843adcaa2dcf75823d79c102cb44fba08a1937fa390678a681" exitCode=0
Jul 28 19:40:27.967611 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:27.967242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t" event={"ID":"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5","Type":"ContainerDied","Data":"7724ac99f4b871843adcaa2dcf75823d79c102cb44fba08a1937fa390678a681"}
Jul 28 19:40:28.974746 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:28.974698    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t" event={"ID":"41dd91d3-1775-4fd6-bb7a-e746ab69fbd5","Type":"ContainerStarted","Data":"1c856ddaf6fc086e1de25f09480def5c1c713b89e00686c4f57d14bead85d0a5"}
Jul 28 19:40:28.975158 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:28.974846    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:28.997029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:28.996982    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t" podStartSLOduration=1.818668146 podStartE2EDuration="5.996966403s" totalImagesPullingTime="3.140030847s" totalInitContainerRuntime="1.03826741s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:28.996153576 +0000 UTC m=+312.885030547" watchObservedRunningTime="2026-07-28 19:40:28.996966403 +0000 UTC m=+312.885843370"
Jul 28 19:40:39.981366 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:39.981336    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-85b9t"
Jul 28 19:40:42.781915 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.781877    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"]
Jul 28 19:40:42.788117 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.788094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.791484 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.791463    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:40:42.791599 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.791512    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:40:42.791599 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.791515    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:40:42.791889 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.791873    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 19:40:42.792264 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.792247    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zr7lh\""
Jul 28 19:40:42.817438 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.817407    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"]
Jul 28 19:40:42.880355 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.880323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tmp\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.880525 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.880368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5bt7\" (UniqueName: \"kubernetes.io/projected/ff98df05-5129-4bcb-b8b9-35db618ea6e6-kube-api-access-t5bt7\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.880525 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.880440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tls-certs\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.966498 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.966460    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5p84w"]
Jul 28 19:40:42.969517 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.969497    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:42.971718 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.971694    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 19:40:42.971991 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.971974    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-l6cwd\""
Jul 28 19:40:42.980245 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.980223    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5p84w"]
Jul 28 19:40:42.981144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.981119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tls-certs\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.981263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.981244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tmp\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.981327 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.981281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5bt7\" (UniqueName: \"kubernetes.io/projected/ff98df05-5129-4bcb-b8b9-35db618ea6e6-kube-api-access-t5bt7\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.983933 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.983909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tmp\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:42.984284 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:42.984259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff98df05-5129-4bcb-b8b9-35db618ea6e6-tls-certs\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:43.001073 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.001022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5bt7\" (UniqueName: \"kubernetes.io/projected/ff98df05-5129-4bcb-b8b9-35db618ea6e6-kube-api-access-t5bt7\") pod \"kube-auth-proxy-5f9868ff69-zw7pq\" (UID: \"ff98df05-5129-4bcb-b8b9-35db618ea6e6\") " pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:43.082156 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.082119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.082315 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.082264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd6d9\" (UniqueName: \"kubernetes.io/projected/79098d57-0b49-4fad-951f-891a2ee5d42c-kube-api-access-wd6d9\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.098036 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.098003    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"
Jul 28 19:40:43.183512 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.183477    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.183677 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.183587    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd6d9\" (UniqueName: \"kubernetes.io/projected/79098d57-0b49-4fad-951f-891a2ee5d42c-kube-api-access-wd6d9\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.184195 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:40:43.183862    2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 19:40:43.184195 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:40:43.183935    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert podName:79098d57-0b49-4fad-951f-891a2ee5d42c nodeName:}" failed. No retries permitted until 2026-07-28 19:40:43.683912244 +0000 UTC m=+327.572789209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert") pod "odh-model-controller-7998c67dc4-5p84w" (UID: "79098d57-0b49-4fad-951f-891a2ee5d42c") : secret "odh-model-controller-webhook-cert" not found
Jul 28 19:40:43.197755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.196479    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd6d9\" (UniqueName: \"kubernetes.io/projected/79098d57-0b49-4fad-951f-891a2ee5d42c-kube-api-access-wd6d9\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.244168 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.244138    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq"]
Jul 28 19:40:43.247319 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:40:43.247296    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff98df05_5129_4bcb_b8b9_35db618ea6e6.slice/crio-60f5082b19f0a7588d4f4f52af18e9e9379ac9066a60e4f984e68fb977cd6778 WatchSource:0}: Error finding container 60f5082b19f0a7588d4f4f52af18e9e9379ac9066a60e4f984e68fb977cd6778: Status 404 returned error can't find the container with id 60f5082b19f0a7588d4f4f52af18e9e9379ac9066a60e4f984e68fb977cd6778
Jul 28 19:40:43.687721 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.687673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.690327 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.690299    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79098d57-0b49-4fad-951f-891a2ee5d42c-cert\") pod \"odh-model-controller-7998c67dc4-5p84w\" (UID: \"79098d57-0b49-4fad-951f-891a2ee5d42c\") " pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.880934 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.880895    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:43.903358 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.903327    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:43.906576 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.906554    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:43.909499 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.909020    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 19:40:43.909499 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.909308    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 19:40:43.909499 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.909374    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 19:40:43.909699 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.909576    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 19:40:43.910001 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.909981    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d77fw\""
Jul 28 19:40:43.920915 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:43.920857    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:44.034136 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.034094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq" event={"ID":"ff98df05-5129-4bcb-b8b9-35db618ea6e6","Type":"ContainerStarted","Data":"60f5082b19f0a7588d4f4f52af18e9e9379ac9066a60e4f984e68fb977cd6778"}
Jul 28 19:40:44.038675 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.038615    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5p84w"]
Jul 28 19:40:44.042381 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:40:44.042353    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79098d57_0b49_4fad_951f_891a2ee5d42c.slice/crio-037206e1e210261f38b296db414afc1299ce41c41c936177702be70183b985ba WatchSource:0}: Error finding container 037206e1e210261f38b296db414afc1299ce41c41c936177702be70183b985ba: Status 404 returned error can't find the container with id 037206e1e210261f38b296db414afc1299ce41c41c936177702be70183b985ba
Jul 28 19:40:44.091889 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.091853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092007 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.091897    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxb7\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092007 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.091921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092007 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.091979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092128 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.092029    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092128 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.092105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.092199 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.092149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193302 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193302 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193266    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxb7\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.193523 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.193484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.194047 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.194018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.196674 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.196649    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.196674 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.196664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.196878 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.196720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.197225 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.197205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.200871 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.200830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.201409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.201387    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxb7\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7\") pod \"istiod-openshift-gateway-7b7995d5cd-64mlh\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.220218 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.220194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:44.382444 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:44.382273    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:44.385512 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:40:44.385484    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13ea7e8_f9a5_4932_9a58_8a450b08cf6a.slice/crio-3e8bb29c3a2c17bd75332dc308d34452d34d9811d68295f240c2456a98d90955 WatchSource:0}: Error finding container 3e8bb29c3a2c17bd75332dc308d34452d34d9811d68295f240c2456a98d90955: Status 404 returned error can't find the container with id 3e8bb29c3a2c17bd75332dc308d34452d34d9811d68295f240c2456a98d90955
Jul 28 19:40:45.041264 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:45.041147    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w" event={"ID":"79098d57-0b49-4fad-951f-891a2ee5d42c","Type":"ContainerStarted","Data":"037206e1e210261f38b296db414afc1299ce41c41c936177702be70183b985ba"}
Jul 28 19:40:45.043585 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:45.043555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" event={"ID":"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a","Type":"ContainerStarted","Data":"3e8bb29c3a2c17bd75332dc308d34452d34d9811d68295f240c2456a98d90955"}
Jul 28 19:40:48.757814 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:48.757775    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:40:48.758089 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:48.757856    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:40:49.062237 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.062132    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq" event={"ID":"ff98df05-5129-4bcb-b8b9-35db618ea6e6","Type":"ContainerStarted","Data":"67bc51b6c4f3825a2e40db06bf46a51d0d3b79541efd6d41079dd9cfbc29696c"}
Jul 28 19:40:49.063684 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.063653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" event={"ID":"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a","Type":"ContainerStarted","Data":"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"}
Jul 28 19:40:49.063889 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.063864    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:49.066289 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.065610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w" event={"ID":"79098d57-0b49-4fad-951f-891a2ee5d42c","Type":"ContainerStarted","Data":"dc8ac69122cc6158e28f201c8f14f7d97f46031ccb55e74ed5b095a0f09ac0a9"}
Jul 28 19:40:49.066289 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.066112    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:40:49.066289 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.066183    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-64mlh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:40:49.066289 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.066220    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:40:49.078852 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.078797    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f9868ff69-zw7pq" podStartSLOduration=1.577538366 podStartE2EDuration="7.078779005s" totalImagesPullingTime="5.501240639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:49.077573043 +0000 UTC m=+332.966450015" watchObservedRunningTime="2026-07-28 19:40:49.078779005 +0000 UTC m=+332.967655978"
Jul 28 19:40:49.093686 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.093631    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w" podStartSLOduration=2.374441695 podStartE2EDuration="7.093610571s" totalImagesPullingTime="4.719168876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:49.092043248 +0000 UTC m=+332.980920221" watchObservedRunningTime="2026-07-28 19:40:49.093610571 +0000 UTC m=+332.982487544"
Jul 28 19:40:49.114482 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:49.114335    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" podStartSLOduration=1.7450736999999998 podStartE2EDuration="6.114314211s" totalImagesPullingTime="4.369240511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:49.113381079 +0000 UTC m=+333.002258070" watchObservedRunningTime="2026-07-28 19:40:49.114314211 +0000 UTC m=+333.003191184"
Jul 28 19:40:50.070772 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:50.070719    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-64mlh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:40:50.071143 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:50.070809    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:40:53.031996 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.031963    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"]
Jul 28 19:40:53.035546 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.035524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.052079 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.052054    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"]
Jul 28 19:40:53.071065 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.071039    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:53.079827 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079805    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.079926 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.079926 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079871    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.079926 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.080062 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.080062 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.080062 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.079984    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqmc\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-kube-api-access-mtqmc\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181070 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181070 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqmc\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-kube-api-access-mtqmc\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.181596 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.181418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.182217 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.182183    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.183573 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.183546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.183758 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.183722    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.183975 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.183954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.184038 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.183957    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.189963 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.189945    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqmc\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-kube-api-access-mtqmc\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.190110 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.190093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9359fcb2-2da7-4c37-8ad6-c998f2830dbf-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2n7lp\" (UID: \"9359fcb2-2da7-4c37-8ad6-c998f2830dbf\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.345252 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.345216    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:53.478364 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.478340    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"]
Jul 28 19:40:53.480706 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:40:53.480673    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9359fcb2_2da7_4c37_8ad6_c998f2830dbf.slice/crio-aa145ce85a0113b8850549d74e39b7308ef7ae1c504731d187b88ab7555fe6af WatchSource:0}: Error finding container aa145ce85a0113b8850549d74e39b7308ef7ae1c504731d187b88ab7555fe6af: Status 404 returned error can't find the container with id aa145ce85a0113b8850549d74e39b7308ef7ae1c504731d187b88ab7555fe6af
Jul 28 19:40:53.484370 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.484321    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:40:53.484467 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:53.484410    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:40:54.089388 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.089337    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp" event={"ID":"9359fcb2-2da7-4c37-8ad6-c998f2830dbf","Type":"ContainerStarted","Data":"9746d68781b4efdc781efbf662d8b88b4886850e9bb108c374b39245b35db1e9"}
Jul 28 19:40:54.089863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.089418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp" event={"ID":"9359fcb2-2da7-4c37-8ad6-c998f2830dbf","Type":"ContainerStarted","Data":"aa145ce85a0113b8850549d74e39b7308ef7ae1c504731d187b88ab7555fe6af"}
Jul 28 19:40:54.089863 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.089561    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:54.091522 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.091496    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-2n7lp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:40:54.091651 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.091552    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp" podUID="9359fcb2-2da7-4c37-8ad6-c998f2830dbf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:40:54.133923 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:54.133880    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp" podStartSLOduration=1.133867283 podStartE2EDuration="1.133867283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:54.131879803 +0000 UTC m=+338.020756775" watchObservedRunningTime="2026-07-28 19:40:54.133867283 +0000 UTC m=+338.022744253"
Jul 28 19:40:55.093659 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.093634    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2n7lp"
Jul 28 19:40:55.149520 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.149487    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:55.149719 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.149693    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery" containerID="cri-o://b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4" gracePeriod=30
Jul 28 19:40:55.387132 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.387104    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:55.502023 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.501996    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502209 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502030    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfxb7\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502209 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502048    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502209 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502087    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502369 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502298    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502369 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502348    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502474 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502387    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert\") pod \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\" (UID: \"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a\") "
Jul 28 19:40:55.502749 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502537    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:40:55.502749 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.502709    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-ca-configmap\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.504496 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504451    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts" (OuterVolumeSpecName: "cacerts") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:40:55.504496 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504488    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7" (OuterVolumeSpecName: "kube-api-access-xfxb7") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "kube-api-access-xfxb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:40:55.504643 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504620    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:40:55.504680 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504659    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:40:55.504814 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504794    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs" (OuterVolumeSpecName: "local-certs") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:40:55.504861 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.504809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" (UID: "f13ea7e8-f9a5-4932-9a58-8a450b08cf6a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:40:55.604010 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.603980    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-cacerts\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.604010 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.604004    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-kubeconfig\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.604010 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.604015    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-local-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.604206 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.604023    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-csr-dns-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.604206 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.604031    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:55.604206 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:55.604040    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfxb7\" (UniqueName: \"kubernetes.io/projected/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a-kube-api-access-xfxb7\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:40:56.098275 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.098238    2575 generic.go:356] "Generic (PLEG): container finished" podID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerID="b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4" exitCode=0
Jul 28 19:40:56.098693 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.098287    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"
Jul 28 19:40:56.098693 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.098320    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" event={"ID":"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a","Type":"ContainerDied","Data":"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"}
Jul 28 19:40:56.098693 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.098365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh" event={"ID":"f13ea7e8-f9a5-4932-9a58-8a450b08cf6a","Type":"ContainerDied","Data":"3e8bb29c3a2c17bd75332dc308d34452d34d9811d68295f240c2456a98d90955"}
Jul 28 19:40:56.098693 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.098399    2575 scope.go:117] "RemoveContainer" containerID="b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"
Jul 28 19:40:56.106792 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.106773    2575 scope.go:117] "RemoveContainer" containerID="b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"
Jul 28 19:40:56.107068 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:40:56.107045    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4\": container with ID starting with b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4 not found: ID does not exist" containerID="b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"
Jul 28 19:40:56.107141 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.107075    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4"} err="failed to get container status \"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4\": rpc error: code = NotFound desc = could not find container \"b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4\": container with ID starting with b6478c2809ca37db34a49394e0def61be95f5c50c6369677397af3f235bc16d4 not found: ID does not exist"
Jul 28 19:40:56.121004 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.120979    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:56.130657 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.130637    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-64mlh"]
Jul 28 19:40:56.694326 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:40:56.694297    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" path="/var/lib/kubelet/pods/f13ea7e8-f9a5-4932-9a58-8a450b08cf6a/volumes"
Jul 28 19:41:01.076918 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:01.076886    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5p84w"
Jul 28 19:41:43.832775 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.832742    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:43.833205 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.833068    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery"
Jul 28 19:41:43.833205 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.833080    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery"
Jul 28 19:41:43.833205 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.833139    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13ea7e8-f9a5-4932-9a58-8a450b08cf6a" containerName="discovery"
Jul 28 19:41:43.835049 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.835030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:43.837768 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.837713    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2x2v4\""
Jul 28 19:41:43.838126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.838103    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:41:43.838306 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.838288    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:41:43.858624 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.858600    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:43.922533 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:43.922502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dwv4\" (UniqueName: \"kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4\") pod \"kuadrant-operator-catalog-rwrbr\" (UID: \"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f\") " pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:44.022923 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.022889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dwv4\" (UniqueName: \"kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4\") pod \"kuadrant-operator-catalog-rwrbr\" (UID: \"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f\") " pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:44.031806 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.031784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dwv4\" (UniqueName: \"kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4\") pod \"kuadrant-operator-catalog-rwrbr\" (UID: \"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f\") " pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:44.145263 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.145206    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:44.175257 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.175231    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:44.272250 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.272184    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:44.274404 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:41:44.274377    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8501fdcb_a004_4e6e_a8ee_334e0bf45a0f.slice/crio-393e0e2a98bc3610123054950c0bb7a3eb03bf7867df3d86a37da782d693121c WatchSource:0}: Error finding container 393e0e2a98bc3610123054950c0bb7a3eb03bf7867df3d86a37da782d693121c: Status 404 returned error can't find the container with id 393e0e2a98bc3610123054950c0bb7a3eb03bf7867df3d86a37da782d693121c
Jul 28 19:41:44.292014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.291987    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" event={"ID":"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f","Type":"ContainerStarted","Data":"393e0e2a98bc3610123054950c0bb7a3eb03bf7867df3d86a37da782d693121c"}
Jul 28 19:41:44.394639 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.394608    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2r5b9"]
Jul 28 19:41:44.397620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.397581    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:44.408696 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.408671    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2r5b9"]
Jul 28 19:41:44.427492 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.427467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/98a87f4d-2bda-4cd6-a724-cfa574672365-kube-api-access-nv2sf\") pod \"kuadrant-operator-catalog-2r5b9\" (UID: \"98a87f4d-2bda-4cd6-a724-cfa574672365\") " pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:44.528329 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.528292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/98a87f4d-2bda-4cd6-a724-cfa574672365-kube-api-access-nv2sf\") pod \"kuadrant-operator-catalog-2r5b9\" (UID: \"98a87f4d-2bda-4cd6-a724-cfa574672365\") " pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:44.537362 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.537335    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/98a87f4d-2bda-4cd6-a724-cfa574672365-kube-api-access-nv2sf\") pod \"kuadrant-operator-catalog-2r5b9\" (UID: \"98a87f4d-2bda-4cd6-a724-cfa574672365\") " pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:44.707723 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.707643    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:44.848642 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:44.848608    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2r5b9"]
Jul 28 19:41:44.852278 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:41:44.852248    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a87f4d_2bda_4cd6_a724_cfa574672365.slice/crio-ed70a93fb46be6a29f9da517f1616b0716a16855d1bd6ed18dd002e60da98830 WatchSource:0}: Error finding container ed70a93fb46be6a29f9da517f1616b0716a16855d1bd6ed18dd002e60da98830: Status 404 returned error can't find the container with id ed70a93fb46be6a29f9da517f1616b0716a16855d1bd6ed18dd002e60da98830
Jul 28 19:41:45.297741 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:45.297699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9" event={"ID":"98a87f4d-2bda-4cd6-a724-cfa574672365","Type":"ContainerStarted","Data":"ed70a93fb46be6a29f9da517f1616b0716a16855d1bd6ed18dd002e60da98830"}
Jul 28 19:41:47.308326 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.308289    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9" event={"ID":"98a87f4d-2bda-4cd6-a724-cfa574672365","Type":"ContainerStarted","Data":"5ea19c6c36b8d91ccac9c4fa11df97a4b89b6c765fedc45a3b4d069d4292e1a5"}
Jul 28 19:41:47.309891 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.309860    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" event={"ID":"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f","Type":"ContainerStarted","Data":"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"}
Jul 28 19:41:47.310125 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.309907    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" podUID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" containerName="registry-server" containerID="cri-o://6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78" gracePeriod=2
Jul 28 19:41:47.324462 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.324411    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9" podStartSLOduration=1.712553016 podStartE2EDuration="3.324395906s" totalImagesPullingTime="1.61184289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:47.32370299 +0000 UTC m=+391.212579962" watchObservedRunningTime="2026-07-28 19:41:47.324395906 +0000 UTC m=+391.213272878"
Jul 28 19:41:47.344285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.344241    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" podStartSLOduration=2.157469273 podStartE2EDuration="4.344227598s" totalImagesPullingTime="2.186758325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:47.341795395 +0000 UTC m=+391.230672366" watchObservedRunningTime="2026-07-28 19:41:47.344227598 +0000 UTC m=+391.233104568"
Jul 28 19:41:47.558831 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.558760    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:47.660478 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.660453    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dwv4\" (UniqueName: \"kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4\") pod \"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f\" (UID: \"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f\") "
Jul 28 19:41:47.662528 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.662500    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4" (OuterVolumeSpecName: "kube-api-access-8dwv4") pod "8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" (UID: "8501fdcb-a004-4e6e-a8ee-334e0bf45a0f"). InnerVolumeSpecName "kube-api-access-8dwv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:47.761639 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:47.761414    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dwv4\" (UniqueName: \"kubernetes.io/projected/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f-kube-api-access-8dwv4\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:48.315424 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.315391    2575 generic.go:356] "Generic (PLEG): container finished" podID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" containerID="6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78" exitCode=0
Jul 28 19:41:48.315880 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.315436    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr"
Jul 28 19:41:48.315880 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.315467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" event={"ID":"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f","Type":"ContainerDied","Data":"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"}
Jul 28 19:41:48.315880 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.315522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rwrbr" event={"ID":"8501fdcb-a004-4e6e-a8ee-334e0bf45a0f","Type":"ContainerDied","Data":"393e0e2a98bc3610123054950c0bb7a3eb03bf7867df3d86a37da782d693121c"}
Jul 28 19:41:48.315880 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.315548    2575 scope.go:117] "RemoveContainer" containerID="6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"
Jul 28 19:41:48.324539 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.324520    2575 scope.go:117] "RemoveContainer" containerID="6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"
Jul 28 19:41:48.324826 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:41:48.324802    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78\": container with ID starting with 6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78 not found: ID does not exist" containerID="6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"
Jul 28 19:41:48.324895 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.324839    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78"} err="failed to get container status \"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78\": rpc error: code = NotFound desc = could not find container \"6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78\": container with ID starting with 6672c772cc786e93a5a899777f703d9a2d47e54df1488619cedcbbff2e956c78 not found: ID does not exist"
Jul 28 19:41:48.335600 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.335576    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:48.339401 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.339381    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rwrbr"]
Jul 28 19:41:48.695458 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:48.695414    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" path="/var/lib/kubelet/pods/8501fdcb-a004-4e6e-a8ee-334e0bf45a0f/volumes"
Jul 28 19:41:54.708482 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:54.708450    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:54.708989 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:54.708540    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:54.729541 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:54.729514    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:41:55.369033 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:41:55.369009    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2r5b9"
Jul 28 19:42:15.430698 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.430649    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"]
Jul 28 19:42:15.431178 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.431158    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" containerName="registry-server"
Jul 28 19:42:15.431249 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.431183    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" containerName="registry-server"
Jul 28 19:42:15.431305 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.431261    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8501fdcb-a004-4e6e-a8ee-334e0bf45a0f" containerName="registry-server"
Jul 28 19:42:15.434455 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.434431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:15.437060 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.437042    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-h9879\""
Jul 28 19:42:15.437147 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.437063    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 19:42:15.444062 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.444031    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"]
Jul 28 19:42:15.495155 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.495121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd9lp\" (UniqueName: \"kubernetes.io/projected/4e6af6e7-74a7-41c4-ba25-c45a4545845a-kube-api-access-kd9lp\") pod \"dns-operator-controller-manager-864dd78c4c-npsd5\" (UID: \"4e6af6e7-74a7-41c4-ba25-c45a4545845a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:15.596201 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.596169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd9lp\" (UniqueName: \"kubernetes.io/projected/4e6af6e7-74a7-41c4-ba25-c45a4545845a-kube-api-access-kd9lp\") pod \"dns-operator-controller-manager-864dd78c4c-npsd5\" (UID: \"4e6af6e7-74a7-41c4-ba25-c45a4545845a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:15.615378 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.615347    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd9lp\" (UniqueName: \"kubernetes.io/projected/4e6af6e7-74a7-41c4-ba25-c45a4545845a-kube-api-access-kd9lp\") pod \"dns-operator-controller-manager-864dd78c4c-npsd5\" (UID: \"4e6af6e7-74a7-41c4-ba25-c45a4545845a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:15.745013 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.744929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:15.878498 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:15.878470    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"]
Jul 28 19:42:15.880869 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:15.880840    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6af6e7_74a7_41c4_ba25_c45a4545845a.slice/crio-b501d25dc1e50239e172aa9252643b857530719d0a207c784f5fdceba19cc713 WatchSource:0}: Error finding container b501d25dc1e50239e172aa9252643b857530719d0a207c784f5fdceba19cc713: Status 404 returned error can't find the container with id b501d25dc1e50239e172aa9252643b857530719d0a207c784f5fdceba19cc713
Jul 28 19:42:16.432802 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:16.432765    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5" event={"ID":"4e6af6e7-74a7-41c4-ba25-c45a4545845a","Type":"ContainerStarted","Data":"b501d25dc1e50239e172aa9252643b857530719d0a207c784f5fdceba19cc713"}
Jul 28 19:42:18.057559 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:18.057538    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 19:42:18.443663 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:18.443625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5" event={"ID":"4e6af6e7-74a7-41c4-ba25-c45a4545845a","Type":"ContainerStarted","Data":"d9ac369b0ab270e9b1225eb17b7989fd01f4fa9ae74c3fde845eee83d52d91f1"}
Jul 28 19:42:18.443862 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:18.443715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:20.854852 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.854803    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5" podStartSLOduration=3.682489354 podStartE2EDuration="5.854784992s" totalImagesPullingTime="2.172295638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:18.475123944 +0000 UTC m=+422.364000916" watchObservedRunningTime="2026-07-28 19:42:20.854784992 +0000 UTC m=+424.743661965"
Jul 28 19:42:20.855772 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.855753    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"]
Jul 28 19:42:20.859889 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.859870    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:20.862867 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.862848    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lcnz8\""
Jul 28 19:42:20.880981 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.880956    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"]
Jul 28 19:42:20.935271 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.935236    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcb9k\" (UniqueName: \"kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:20.935420 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:20.935304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.036208 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.036170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.036385 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.036247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcb9k\" (UniqueName: \"kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.036566 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.036548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.054837 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.054813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcb9k\" (UniqueName: \"kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.170403 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.170314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:21.313066 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.312903    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"]
Jul 28 19:42:21.315978 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:21.315952    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4ec974_7082_4158_9ca7_c4103cbd95fb.slice/crio-2f7cd995fc9a4234948fdfab2e1d84c7b98750bfb50a5543e5b500444b5c22bf WatchSource:0}: Error finding container 2f7cd995fc9a4234948fdfab2e1d84c7b98750bfb50a5543e5b500444b5c22bf: Status 404 returned error can't find the container with id 2f7cd995fc9a4234948fdfab2e1d84c7b98750bfb50a5543e5b500444b5c22bf
Jul 28 19:42:21.458424 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:21.458340    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh" event={"ID":"cf4ec974-7082-4158-9ca7-c4103cbd95fb","Type":"ContainerStarted","Data":"2f7cd995fc9a4234948fdfab2e1d84c7b98750bfb50a5543e5b500444b5c22bf"}
Jul 28 19:42:24.221151 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.221119    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"]
Jul 28 19:42:24.223874 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.223849    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:24.227019 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.226994    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kt4hx\""
Jul 28 19:42:24.244984 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.244952    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"]
Jul 28 19:42:24.264043 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.264015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvmnz\" (UniqueName: \"kubernetes.io/projected/df0b661f-828e-4b83-898d-31ab134e1728-kube-api-access-zvmnz\") pod \"authorino-operator-75cc8d5cb7-btsfx\" (UID: \"df0b661f-828e-4b83-898d-31ab134e1728\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:24.364943 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.364908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvmnz\" (UniqueName: \"kubernetes.io/projected/df0b661f-828e-4b83-898d-31ab134e1728-kube-api-access-zvmnz\") pod \"authorino-operator-75cc8d5cb7-btsfx\" (UID: \"df0b661f-828e-4b83-898d-31ab134e1728\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:24.385953 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.385923    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvmnz\" (UniqueName: \"kubernetes.io/projected/df0b661f-828e-4b83-898d-31ab134e1728-kube-api-access-zvmnz\") pod \"authorino-operator-75cc8d5cb7-btsfx\" (UID: \"df0b661f-828e-4b83-898d-31ab134e1728\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:24.535764 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.535665    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:24.671384 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:24.671356    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"]
Jul 28 19:42:24.674004 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:24.673981    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0b661f_828e_4b83_898d_31ab134e1728.slice/crio-76699ccc7b44b4f25b4beeba56a51101df8ac9fc7874a38cf23651610c2549f5 WatchSource:0}: Error finding container 76699ccc7b44b4f25b4beeba56a51101df8ac9fc7874a38cf23651610c2549f5: Status 404 returned error can't find the container with id 76699ccc7b44b4f25b4beeba56a51101df8ac9fc7874a38cf23651610c2549f5
Jul 28 19:42:25.481602 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:25.481562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx" event={"ID":"df0b661f-828e-4b83-898d-31ab134e1728","Type":"ContainerStarted","Data":"76699ccc7b44b4f25b4beeba56a51101df8ac9fc7874a38cf23651610c2549f5"}
Jul 28 19:42:27.033899 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.033860    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"]
Jul 28 19:42:27.036835 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.036798    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:27.039716 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.039693    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zdtj2\""
Jul 28 19:42:27.055199 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.055171    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"]
Jul 28 19:42:27.090683 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.090641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb26l\" (UniqueName: \"kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l\") pod \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" (UID: \"3a7c2c42-b4e2-4f45-8e03-a0fe64031157\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:27.192043 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.192005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb26l\" (UniqueName: \"kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l\") pod \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" (UID: \"3a7c2c42-b4e2-4f45-8e03-a0fe64031157\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:27.203780 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.203748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb26l\" (UniqueName: \"kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l\") pod \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" (UID: \"3a7c2c42-b4e2-4f45-8e03-a0fe64031157\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:27.348802 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:27.348763    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:28.309211 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.309176    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"]
Jul 28 19:42:28.314884 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:28.314849    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7c2c42_b4e2_4f45_8e03_a0fe64031157.slice/crio-ae833d24ea4e318327be9d22d7fddedadbf35355df14daf77eae1886d6982062 WatchSource:0}: Error finding container ae833d24ea4e318327be9d22d7fddedadbf35355df14daf77eae1886d6982062: Status 404 returned error can't find the container with id ae833d24ea4e318327be9d22d7fddedadbf35355df14daf77eae1886d6982062
Jul 28 19:42:28.497322 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.497238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh" event={"ID":"cf4ec974-7082-4158-9ca7-c4103cbd95fb","Type":"ContainerStarted","Data":"d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a"}
Jul 28 19:42:28.497480 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.497338    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:28.498907 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.498874    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx" event={"ID":"df0b661f-828e-4b83-898d-31ab134e1728","Type":"ContainerStarted","Data":"2ec38ca1f7daf65f7cc36db51ba86fff69d86bd8bc8fcf3c1cbc47a6016a0803"}
Jul 28 19:42:28.499064 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.498936    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:28.500159 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.500138    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk" event={"ID":"3a7c2c42-b4e2-4f45-8e03-a0fe64031157","Type":"ContainerStarted","Data":"ae833d24ea4e318327be9d22d7fddedadbf35355df14daf77eae1886d6982062"}
Jul 28 19:42:28.557844 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.557794    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh" podStartSLOduration=1.7147110319999999 podStartE2EDuration="8.557777872s" totalImagesPullingTime="6.84306684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:28.556621737 +0000 UTC m=+432.445498710" watchObservedRunningTime="2026-07-28 19:42:28.557777872 +0000 UTC m=+432.446654842"
Jul 28 19:42:28.588362 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:28.588310    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx" podStartSLOduration=1.103286169 podStartE2EDuration="4.588293939s" totalImagesPullingTime="3.48500777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:28.587099981 +0000 UTC m=+432.475976950" watchObservedRunningTime="2026-07-28 19:42:28.588293939 +0000 UTC m=+432.477170910"
Jul 28 19:42:29.451608 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:29.451573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-npsd5"
Jul 28 19:42:30.511992 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:30.511951    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk" event={"ID":"3a7c2c42-b4e2-4f45-8e03-a0fe64031157","Type":"ContainerStarted","Data":"001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d"}
Jul 28 19:42:30.512359 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:30.512097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:30.532812 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:30.532762    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk" podStartSLOduration=1.951953386 podStartE2EDuration="3.532748s" totalImagesPullingTime="1.580794614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:30.531057153 +0000 UTC m=+434.419934128" watchObservedRunningTime="2026-07-28 19:42:30.532748 +0000 UTC m=+434.421624961"
Jul 28 19:42:39.507488 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:39.507407    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-btsfx"
Jul 28 19:42:39.507869 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:39.507507    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.519083 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.519054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.675156 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.675118    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"]
Jul 28 19:42:41.675327 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.675289    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" containerName="manager" containerID="cri-o://d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a" gracePeriod=2
Jul 28 19:42:41.678183 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.678145    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.679747 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.679703    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"]
Jul 28 19:42:41.695853 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.695824    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"]
Jul 28 19:42:41.696068 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.696044    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" containerName="manager" containerID="cri-o://001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d" gracePeriod=2
Jul 28 19:42:41.703590 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.703558    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.703995 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.703973    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"]
Jul 28 19:42:41.709814 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.709778    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.716382 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.716356    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:42:41.716875 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.716857    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" containerName="manager"
Jul 28 19:42:41.716952 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.716877    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" containerName="manager"
Jul 28 19:42:41.716952 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.716900    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" containerName="manager"
Jul 28 19:42:41.716952 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.716910    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" containerName="manager"
Jul 28 19:42:41.717108 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.717011    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" containerName="manager"
Jul 28 19:42:41.717108 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.717029    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" containerName="manager"
Jul 28 19:42:41.719249 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.719229    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.735380 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.735327    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"]
Jul 28 19:42:41.737872 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.737853    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:41.748902 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.748840    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:42:41.760398 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.760375    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"]
Jul 28 19:42:41.789889 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.789827    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.799663 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.799629    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.801438 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.801410    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.805188 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.805161    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.819360 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.819331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2t22\" (UniqueName: \"kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.819501 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.819372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nx5g\" (UniqueName: \"kubernetes.io/projected/d19913b3-f734-4b87-b076-524d7d484f8a-kube-api-access-2nx5g\") pod \"limitador-operator-controller-manager-859d7b78bd-gpqr2\" (UID: \"d19913b3-f734-4b87-b076-524d7d484f8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:41.819501 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.819396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.920433 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.920400    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.920594 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.920524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2t22\" (UniqueName: \"kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.920594 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.920569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nx5g\" (UniqueName: \"kubernetes.io/projected/d19913b3-f734-4b87-b076-524d7d484f8a-kube-api-access-2nx5g\") pod \"limitador-operator-controller-manager-859d7b78bd-gpqr2\" (UID: \"d19913b3-f734-4b87-b076-524d7d484f8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:41.920837 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.920809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.935056 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.935031    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nx5g\" (UniqueName: \"kubernetes.io/projected/d19913b3-f734-4b87-b076-524d7d484f8a-kube-api-access-2nx5g\") pod \"limitador-operator-controller-manager-859d7b78bd-gpqr2\" (UID: \"d19913b3-f734-4b87-b076-524d7d484f8a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:41.937864 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.937842    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.943352 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.943326    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.945666 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.945645    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.947588 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.947566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2t22\" (UniqueName: \"kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lz48g\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:41.948650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:41.948635    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:41.950524 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.950501    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:41.952511 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:41.952483    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:42.021486 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.021457    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb26l\" (UniqueName: \"kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l\") pod \"3a7c2c42-b4e2-4f45-8e03-a0fe64031157\" (UID: \"3a7c2c42-b4e2-4f45-8e03-a0fe64031157\") "
Jul 28 19:42:42.021620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.021514    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume\") pod \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") "
Jul 28 19:42:42.021620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.021553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcb9k\" (UniqueName: \"kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k\") pod \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\" (UID: \"cf4ec974-7082-4158-9ca7-c4103cbd95fb\") "
Jul 28 19:42:42.022032 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.022007    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cf4ec974-7082-4158-9ca7-c4103cbd95fb" (UID: "cf4ec974-7082-4158-9ca7-c4103cbd95fb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:42:42.023490 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.023467    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k" (OuterVolumeSpecName: "kube-api-access-kcb9k") pod "cf4ec974-7082-4158-9ca7-c4103cbd95fb" (UID: "cf4ec974-7082-4158-9ca7-c4103cbd95fb"). InnerVolumeSpecName "kube-api-access-kcb9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:42:42.023567 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.023496    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l" (OuterVolumeSpecName: "kube-api-access-vb26l") pod "3a7c2c42-b4e2-4f45-8e03-a0fe64031157" (UID: "3a7c2c42-b4e2-4f45-8e03-a0fe64031157"). InnerVolumeSpecName "kube-api-access-vb26l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:42:42.097885 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.097857    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:42.103831 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.103810    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:42.122745 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.122701    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb26l\" (UniqueName: \"kubernetes.io/projected/3a7c2c42-b4e2-4f45-8e03-a0fe64031157-kube-api-access-vb26l\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:42:42.122745 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.122743    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ec974-7082-4158-9ca7-c4103cbd95fb-extensions-socket-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:42:42.122870 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.122760    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcb9k\" (UniqueName: \"kubernetes.io/projected/cf4ec974-7082-4158-9ca7-c4103cbd95fb-kube-api-access-kcb9k\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:42:42.307395 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:42.307361    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19913b3_f734_4b87_b076_524d7d484f8a.slice/crio-5a321062aeb3002766367e8185b77707fdb8def625d37a927178bbd0d1ae8b87 WatchSource:0}: Error finding container 5a321062aeb3002766367e8185b77707fdb8def625d37a927178bbd0d1ae8b87: Status 404 returned error can't find the container with id 5a321062aeb3002766367e8185b77707fdb8def625d37a927178bbd0d1ae8b87
Jul 28 19:42:42.308193 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:42:42.308170    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1ac3aa_dc2e_4a6a_aabe_ca7669c2105b.slice/crio-d56b465407437ba58ce887030427802bb6a188dd5768fec03e299d621d3d3b0b WatchSource:0}: Error finding container d56b465407437ba58ce887030427802bb6a188dd5768fec03e299d621d3d3b0b: Status 404 returned error can't find the container with id d56b465407437ba58ce887030427802bb6a188dd5768fec03e299d621d3d3b0b
Jul 28 19:42:42.311629 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.311609    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"]
Jul 28 19:42:42.313380 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.313357    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:42:42.575603 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.575572    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" containerID="d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a" exitCode=0
Jul 28 19:42:42.575603 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.575623    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:42.576150 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.575672    2575 scope.go:117] "RemoveContainer" containerID="d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a"
Jul 28 19:42:42.577423 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.577397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" event={"ID":"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b","Type":"ContainerStarted","Data":"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"}
Jul 28 19:42:42.577534 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.577450    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" event={"ID":"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b","Type":"ContainerStarted","Data":"d56b465407437ba58ce887030427802bb6a188dd5768fec03e299d621d3d3b0b"}
Jul 28 19:42:42.577534 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.577479    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:42:42.582347 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.582315    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:42.582939 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.582909    2575 generic.go:356] "Generic (PLEG): container finished" podID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" containerID="001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d" exitCode=0
Jul 28 19:42:42.583093 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.583075    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:42.586152 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.586063    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:42.587856 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.587828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2" event={"ID":"d19913b3-f734-4b87-b076-524d7d484f8a","Type":"ContainerStarted","Data":"f8effe711f4d5cfa6b47f433655493b94a37b7fe4bd32bde6983f5529e90ed03"}
Jul 28 19:42:42.588038 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.588012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2" event={"ID":"d19913b3-f734-4b87-b076-524d7d484f8a","Type":"ContainerStarted","Data":"5a321062aeb3002766367e8185b77707fdb8def625d37a927178bbd0d1ae8b87"}
Jul 28 19:42:42.588175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.588053    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:42.591161 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.591146    2575 scope.go:117] "RemoveContainer" containerID="d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a"
Jul 28 19:42:42.591407 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.591388    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a\": container with ID starting with d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a not found: ID does not exist" containerID="d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a"
Jul 28 19:42:42.591464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.591413    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a"} err="failed to get container status \"d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a\": rpc error: code = NotFound desc = could not find container \"d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a\": container with ID starting with d161804e51f385f8dd951644f8cc8e3b506b2386b39403db051e10239272921a not found: ID does not exist"
Jul 28 19:42:42.591464 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.591430    2575 scope.go:117] "RemoveContainer" containerID="001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d"
Jul 28 19:42:42.599163 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.599148    2575 scope.go:117] "RemoveContainer" containerID="001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d"
Jul 28 19:42:42.599432 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.599416    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d\": container with ID starting with 001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d not found: ID does not exist" containerID="001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d"
Jul 28 19:42:42.599481 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.599438    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d"} err="failed to get container status \"001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d\": rpc error: code = NotFound desc = could not find container \"001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d\": container with ID starting with 001bc5c5df57c95099a33afb262b41a55f1bffd6853448c272f1eaa46aae681d not found: ID does not exist"
Jul 28 19:42:42.627671 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.627632    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" podStartSLOduration=1.6276180120000001 podStartE2EDuration="1.627618012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:42.626807897 +0000 UTC m=+446.515684867" watchObservedRunningTime="2026-07-28 19:42:42.627618012 +0000 UTC m=+446.516494982"
Jul 28 19:42:42.629738 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.629705    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:42.631822 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.631789    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:42.642125 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.642098    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8frh\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8frh"
Jul 28 19:42:42.644251 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:42:42.644233    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j4xtk\" is forbidden: User \"system:node:ip-10-0-130-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-70.ec2.internal' and this object" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j4xtk"
Jul 28 19:42:42.691526 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.691473    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2" podStartSLOduration=1.691453568 podStartE2EDuration="1.691453568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:42.689908855 +0000 UTC m=+446.578785826" watchObservedRunningTime="2026-07-28 19:42:42.691453568 +0000 UTC m=+446.580330541"
Jul 28 19:42:42.696107 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.696084    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a7c2c42-b4e2-4f45-8e03-a0fe64031157" path="/var/lib/kubelet/pods/3a7c2c42-b4e2-4f45-8e03-a0fe64031157/volumes"
Jul 28 19:42:42.696471 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:42.696420    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf4ec974-7082-4158-9ca7-c4103cbd95fb" path="/var/lib/kubelet/pods/cf4ec974-7082-4158-9ca7-c4103cbd95fb/volumes"
Jul 28 19:42:53.596828 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:53.596795    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gpqr2"
Jul 28 19:42:53.597297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:42:53.596864    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:43:10.159032 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.158990    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:43:10.159482 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.159282    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" podUID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" containerName="manager" containerID="cri-o://9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1" gracePeriod=10
Jul 28 19:43:10.408799 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.408773    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:43:10.567381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.567294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2t22\" (UniqueName: \"kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22\") pod \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") "
Jul 28 19:43:10.567381 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.567346    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume\") pod \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\" (UID: \"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b\") "
Jul 28 19:43:10.567775 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.567718    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" (UID: "bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:10.569367 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.569342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22" (OuterVolumeSpecName: "kube-api-access-f2t22") pod "bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" (UID: "bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b"). InnerVolumeSpecName "kube-api-access-f2t22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:10.667957 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.667918    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2t22\" (UniqueName: \"kubernetes.io/projected/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-kube-api-access-f2t22\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:10.667957 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.667951    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b-extensions-socket-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:10.711567 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.711539    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" containerID="9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1" exitCode=0
Jul 28 19:43:10.711707 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.711582    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"
Jul 28 19:43:10.711707 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.711607    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" event={"ID":"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b","Type":"ContainerDied","Data":"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"}
Jul 28 19:43:10.711707 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.711652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g" event={"ID":"bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b","Type":"ContainerDied","Data":"d56b465407437ba58ce887030427802bb6a188dd5768fec03e299d621d3d3b0b"}
Jul 28 19:43:10.711707 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.711674    2575 scope.go:117] "RemoveContainer" containerID="9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"
Jul 28 19:43:10.719914 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.719899    2575 scope.go:117] "RemoveContainer" containerID="9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"
Jul 28 19:43:10.720180 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:43:10.720160    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1\": container with ID starting with 9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1 not found: ID does not exist" containerID="9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"
Jul 28 19:43:10.720228 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.720188    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1"} err="failed to get container status \"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1\": rpc error: code = NotFound desc = could not find container \"9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1\": container with ID starting with 9fdcb67b71e232a6b0c5be00c0da961662fac41ad11f19029b8a440c982802b1 not found: ID does not exist"
Jul 28 19:43:10.728599 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.728576    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:43:10.735198 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:10.735176    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lz48g"]
Jul 28 19:43:12.694549 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:12.694515    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" path="/var/lib/kubelet/pods/bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b/volumes"
Jul 28 19:43:26.563906 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.563869    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"]
Jul 28 19:43:26.564518 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.564384    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" containerName="manager"
Jul 28 19:43:26.564518 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.564404    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" containerName="manager"
Jul 28 19:43:26.564518 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.564503    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1ac3aa-dc2e-4a6a-aabe-ca7669c2105b" containerName="manager"
Jul 28 19:43:26.567803 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.567781    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.569862 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.569841    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mkw6c\""
Jul 28 19:43:26.579635 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.579610    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"]
Jul 28 19:43:26.603931 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.603894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.603931 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.603930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5nvh\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-kube-api-access-s5nvh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.603951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.603975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd39ec7-ad59-4d89-bab6-22dc6504b258-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.604000    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.604024    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.604056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.604120    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.604175 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.604160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705259 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705259 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5nvh\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-kube-api-access-s5nvh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705295    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd39ec7-ad59-4d89-bab6-22dc6504b258-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705479 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705794 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705794 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705641    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705794 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705675    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705794 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.705943 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.705842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.706140 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.706117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd39ec7-ad59-4d89-bab6-22dc6504b258-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.713882 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.708093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.713882 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.708661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.715837 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.715813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.716309 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.716291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5nvh\" (UniqueName: \"kubernetes.io/projected/efd39ec7-ad59-4d89-bab6-22dc6504b258-kube-api-access-s5nvh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lcz4q\" (UID: \"efd39ec7-ad59-4d89-bab6-22dc6504b258\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:26.877864 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:26.877814    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:27.009416 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:27.009387    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"]
Jul 28 19:43:27.011514 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:43:27.011487    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd39ec7_ad59_4d89_bab6_22dc6504b258.slice/crio-2aef5e64ce146aa9f4cac137d6db04c12da7b1a81ce29526ac37b2a5430a2648 WatchSource:0}: Error finding container 2aef5e64ce146aa9f4cac137d6db04c12da7b1a81ce29526ac37b2a5430a2648: Status 404 returned error can't find the container with id 2aef5e64ce146aa9f4cac137d6db04c12da7b1a81ce29526ac37b2a5430a2648
Jul 28 19:43:27.789699 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:27.789656    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" event={"ID":"efd39ec7-ad59-4d89-bab6-22dc6504b258","Type":"ContainerStarted","Data":"2aef5e64ce146aa9f4cac137d6db04c12da7b1a81ce29526ac37b2a5430a2648"}
Jul 28 19:43:30.157973 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.157925    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:43:30.158355 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.158024    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:43:30.158355 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.158075    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:43:30.805192 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.805158    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" event={"ID":"efd39ec7-ad59-4d89-bab6-22dc6504b258","Type":"ContainerStarted","Data":"b868b0e4c86b8361716a6e51f5e13270655c8bbee2edbf4a3aa6d5a5a0896e11"}
Jul 28 19:43:30.827536 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.827489    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" podStartSLOduration=1.683248458 podStartE2EDuration="4.827472821s" totalImagesPullingTime="3.144224363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:30.827314845 +0000 UTC m=+494.716191864" watchObservedRunningTime="2026-07-28 19:43:30.827472821 +0000 UTC m=+494.716349791"
Jul 28 19:43:30.878810 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:30.878776    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:31.883441 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.883412    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:31.951487 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.951450    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:31.953709 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.953694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:31.956006 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.955969    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 19:43:31.956006 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.956002    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6gsrx\""
Jul 28 19:43:31.963157 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:31.963132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:32.052788 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.052759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.052949 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.052878    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knk4p\" (UniqueName: \"kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.067651 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.067619    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:32.153425 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.153341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knk4p\" (UniqueName: \"kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.153425 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.153416    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.154250 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.154228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.164299 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.164263    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knk4p\" (UniqueName: \"kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p\") pod \"limitador-limitador-64c94fd64f-kb94m\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.264103 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.264068    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:32.396043 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.395969    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:32.398306 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:43:32.398283    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87eb1b4e_6532_4675_9fbf_ad384ad597cd.slice/crio-69d7e2196cfa1aa94dbaaf4a2fde7c96578829217269034e1251560d9f7cd0af WatchSource:0}: Error finding container 69d7e2196cfa1aa94dbaaf4a2fde7c96578829217269034e1251560d9f7cd0af: Status 404 returned error can't find the container with id 69d7e2196cfa1aa94dbaaf4a2fde7c96578829217269034e1251560d9f7cd0af
Jul 28 19:43:32.751344 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.751295    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:32.754980 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.754957    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:32.757236 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.757208    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s7jjx\""
Jul 28 19:43:32.761319 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.761295    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:32.816058 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.816022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" event={"ID":"87eb1b4e-6532-4675-9fbf-ad384ad597cd","Type":"ContainerStarted","Data":"69d7e2196cfa1aa94dbaaf4a2fde7c96578829217269034e1251560d9f7cd0af"}
Jul 28 19:43:32.816282 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.816265    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:32.817161 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.817145    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:43:32.858297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.858267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb24m\" (UniqueName: \"kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m\") pod \"authorino-58cd955675-ccsrc\" (UID: \"d36a86c0-9657-4309-a580-5c1f6ce79123\") " pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:32.959829 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.959783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb24m\" (UniqueName: \"kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m\") pod \"authorino-58cd955675-ccsrc\" (UID: \"d36a86c0-9657-4309-a580-5c1f6ce79123\") " pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:32.976849 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:32.976789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb24m\" (UniqueName: \"kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m\") pod \"authorino-58cd955675-ccsrc\" (UID: \"d36a86c0-9657-4309-a580-5c1f6ce79123\") " pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:33.080636 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:33.080598    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:33.241778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:33.241744    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:33.245412 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:43:33.245353    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd36a86c0_9657_4309_a580_5c1f6ce79123.slice/crio-aeceb98fd25f37ec78f9e13e6e3dce6ff386180a8987e88d607a17ddf9461c22 WatchSource:0}: Error finding container aeceb98fd25f37ec78f9e13e6e3dce6ff386180a8987e88d607a17ddf9461c22: Status 404 returned error can't find the container with id aeceb98fd25f37ec78f9e13e6e3dce6ff386180a8987e88d607a17ddf9461c22
Jul 28 19:43:33.823750 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:33.823698    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ccsrc" event={"ID":"d36a86c0-9657-4309-a580-5c1f6ce79123","Type":"ContainerStarted","Data":"aeceb98fd25f37ec78f9e13e6e3dce6ff386180a8987e88d607a17ddf9461c22"}
Jul 28 19:43:35.837744 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:35.837688    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" event={"ID":"87eb1b4e-6532-4675-9fbf-ad384ad597cd","Type":"ContainerStarted","Data":"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"}
Jul 28 19:43:35.838187 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:35.837908    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:35.859986 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:35.859918    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" podStartSLOduration=2.126328828 podStartE2EDuration="4.859898132s" totalImagesPullingTime="2.733569304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:35.856901417 +0000 UTC m=+499.745778389" watchObservedRunningTime="2026-07-28 19:43:35.859898132 +0000 UTC m=+499.748775104"
Jul 28 19:43:37.849770 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:37.849707    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ccsrc" event={"ID":"d36a86c0-9657-4309-a580-5c1f6ce79123","Type":"ContainerStarted","Data":"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"}
Jul 28 19:43:37.864272 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:37.864218    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ccsrc" podStartSLOduration=2.217775603 podStartE2EDuration="5.864203235s" totalImagesPullingTime="3.646427632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:37.863642352 +0000 UTC m=+501.752519323" watchObservedRunningTime="2026-07-28 19:43:37.864203235 +0000 UTC m=+501.753080206"
Jul 28 19:43:37.972447 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:37.972413    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:39.859354 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:39.859310    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ccsrc" podUID="d36a86c0-9657-4309-a580-5c1f6ce79123" containerName="authorino" containerID="cri-o://0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a" gracePeriod=30
Jul 28 19:43:40.101679 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.101656    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:40.225916 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.225841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb24m\" (UniqueName: \"kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m\") pod \"d36a86c0-9657-4309-a580-5c1f6ce79123\" (UID: \"d36a86c0-9657-4309-a580-5c1f6ce79123\") "
Jul 28 19:43:40.227709 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.227669    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m" (OuterVolumeSpecName: "kube-api-access-hb24m") pod "d36a86c0-9657-4309-a580-5c1f6ce79123" (UID: "d36a86c0-9657-4309-a580-5c1f6ce79123"). InnerVolumeSpecName "kube-api-access-hb24m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:40.327297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.327258    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb24m\" (UniqueName: \"kubernetes.io/projected/d36a86c0-9657-4309-a580-5c1f6ce79123-kube-api-access-hb24m\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:40.865256 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.865223    2575 generic.go:356] "Generic (PLEG): container finished" podID="d36a86c0-9657-4309-a580-5c1f6ce79123" containerID="0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a" exitCode=0
Jul 28 19:43:40.865650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.865266    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ccsrc"
Jul 28 19:43:40.865650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.865308    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ccsrc" event={"ID":"d36a86c0-9657-4309-a580-5c1f6ce79123","Type":"ContainerDied","Data":"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"}
Jul 28 19:43:40.865650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.865349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ccsrc" event={"ID":"d36a86c0-9657-4309-a580-5c1f6ce79123","Type":"ContainerDied","Data":"aeceb98fd25f37ec78f9e13e6e3dce6ff386180a8987e88d607a17ddf9461c22"}
Jul 28 19:43:40.865650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.865372    2575 scope.go:117] "RemoveContainer" containerID="0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"
Jul 28 19:43:40.873496 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.873476    2575 scope.go:117] "RemoveContainer" containerID="0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"
Jul 28 19:43:40.873764 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:43:40.873744    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a\": container with ID starting with 0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a not found: ID does not exist" containerID="0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"
Jul 28 19:43:40.873819 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.873772    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a"} err="failed to get container status \"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a\": rpc error: code = NotFound desc = could not find container \"0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a\": container with ID starting with 0e67bf1b0b2a2e74eef38aad3c1da0529144d6c6811e92c9ecae7cfb45c3693a not found: ID does not exist"
Jul 28 19:43:40.881521 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.881499    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:40.885275 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:40.885251    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ccsrc"]
Jul 28 19:43:42.695194 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:42.695158    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d36a86c0-9657-4309-a580-5c1f6ce79123" path="/var/lib/kubelet/pods/d36a86c0-9657-4309-a580-5c1f6ce79123/volumes"
Jul 28 19:43:46.844814 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:46.844780    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:48.990974 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:48.990942    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:48.991372 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:48.991128    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" podUID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" containerName="limitador" containerID="cri-o://226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f" gracePeriod=30
Jul 28 19:43:49.543059 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.543033    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:49.710156 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.710120    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file\") pod \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") "
Jul 28 19:43:49.710322 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.710220    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knk4p\" (UniqueName: \"kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p\") pod \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\" (UID: \"87eb1b4e-6532-4675-9fbf-ad384ad597cd\") "
Jul 28 19:43:49.710581 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.710543    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file" (OuterVolumeSpecName: "config-file") pod "87eb1b4e-6532-4675-9fbf-ad384ad597cd" (UID: "87eb1b4e-6532-4675-9fbf-ad384ad597cd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:49.712221 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.712203    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p" (OuterVolumeSpecName: "kube-api-access-knk4p") pod "87eb1b4e-6532-4675-9fbf-ad384ad597cd" (UID: "87eb1b4e-6532-4675-9fbf-ad384ad597cd"). InnerVolumeSpecName "kube-api-access-knk4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:49.811859 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.811823    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knk4p\" (UniqueName: \"kubernetes.io/projected/87eb1b4e-6532-4675-9fbf-ad384ad597cd-kube-api-access-knk4p\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:49.811859 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.811855    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87eb1b4e-6532-4675-9fbf-ad384ad597cd-config-file\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:49.908215 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.908184    2575 generic.go:356] "Generic (PLEG): container finished" podID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" containerID="226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f" exitCode=0
Jul 28 19:43:49.908353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.908225    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m"
Jul 28 19:43:49.908353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.908272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" event={"ID":"87eb1b4e-6532-4675-9fbf-ad384ad597cd","Type":"ContainerDied","Data":"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"}
Jul 28 19:43:49.908353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.908319    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kb94m" event={"ID":"87eb1b4e-6532-4675-9fbf-ad384ad597cd","Type":"ContainerDied","Data":"69d7e2196cfa1aa94dbaaf4a2fde7c96578829217269034e1251560d9f7cd0af"}
Jul 28 19:43:49.908353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.908344    2575 scope.go:117] "RemoveContainer" containerID="226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"
Jul 28 19:43:49.916715 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.916698    2575 scope.go:117] "RemoveContainer" containerID="226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"
Jul 28 19:43:49.916979 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:43:49.916961    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f\": container with ID starting with 226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f not found: ID does not exist" containerID="226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"
Jul 28 19:43:49.917025 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.916987    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f"} err="failed to get container status \"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f\": rpc error: code = NotFound desc = could not find container \"226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f\": container with ID starting with 226706865cce754793c227fe0897a0a93b29cdc8a906077e2632e19f5029494f not found: ID does not exist"
Jul 28 19:43:49.928292 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.928266    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:49.931476 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:49.931454    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kb94m"]
Jul 28 19:43:50.695778 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:50.695721    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" path="/var/lib/kubelet/pods/87eb1b4e-6532-4675-9fbf-ad384ad597cd/volumes"
Jul 28 19:43:53.731384 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.731348    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"]
Jul 28 19:43:53.731903 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.731882    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" containerName="limitador"
Jul 28 19:43:53.731984 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.731907    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" containerName="limitador"
Jul 28 19:43:53.731984 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.731929    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36a86c0-9657-4309-a580-5c1f6ce79123" containerName="authorino"
Jul 28 19:43:53.731984 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.731938    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36a86c0-9657-4309-a580-5c1f6ce79123" containerName="authorino"
Jul 28 19:43:53.735760 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.733941    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87eb1b4e-6532-4675-9fbf-ad384ad597cd" containerName="limitador"
Jul 28 19:43:53.735760 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.733971    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36a86c0-9657-4309-a580-5c1f6ce79123" containerName="authorino"
Jul 28 19:43:53.741595 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.740717    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.743144 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.743117    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"]
Jul 28 19:43:53.743318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.743216    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:43:53.743434 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.743320    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:43:53.743434 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.743324    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6hngb\""
Jul 28 19:43:53.744029 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.743980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 19:43:53.849743 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.849702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59h4p\" (UniqueName: \"kubernetes.io/projected/b2a2923b-d563-47cc-88b1-b86ce977f6d0-kube-api-access-59h4p\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.849899 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.849770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a2923b-d563-47cc-88b1-b86ce977f6d0-data\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.950298 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.950270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59h4p\" (UniqueName: \"kubernetes.io/projected/b2a2923b-d563-47cc-88b1-b86ce977f6d0-kube-api-access-59h4p\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.950430 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.950314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a2923b-d563-47cc-88b1-b86ce977f6d0-data\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.950630 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.950615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b2a2923b-d563-47cc-88b1-b86ce977f6d0-data\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:53.959165 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:53.959140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59h4p\" (UniqueName: \"kubernetes.io/projected/b2a2923b-d563-47cc-88b1-b86ce977f6d0-kube-api-access-59h4p\") pod \"postgres-69fb4594cf-6rwkl\" (UID: \"b2a2923b-d563-47cc-88b1-b86ce977f6d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:54.052421 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:54.052354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:54.181476 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:54.181452    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"]
Jul 28 19:43:54.184083 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:43:54.184054    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a2923b_d563_47cc_88b1_b86ce977f6d0.slice/crio-89103249b4316200f53e19d9eee49659310f35214a7fbea58b4c579d9e17fbf8 WatchSource:0}: Error finding container 89103249b4316200f53e19d9eee49659310f35214a7fbea58b4c579d9e17fbf8: Status 404 returned error can't find the container with id 89103249b4316200f53e19d9eee49659310f35214a7fbea58b4c579d9e17fbf8
Jul 28 19:43:54.933212 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:54.933173    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl" event={"ID":"b2a2923b-d563-47cc-88b1-b86ce977f6d0","Type":"ContainerStarted","Data":"89103249b4316200f53e19d9eee49659310f35214a7fbea58b4c579d9e17fbf8"}
Jul 28 19:43:59.959487 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:59.959447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl" event={"ID":"b2a2923b-d563-47cc-88b1-b86ce977f6d0","Type":"ContainerStarted","Data":"5bb3d28a25859dbe63071d6b016422abcb2989b6a18e8baa02e5dd383b06f74d"}
Jul 28 19:43:59.959875 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:59.959548    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:43:59.975811 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:43:59.975756    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl" podStartSLOduration=1.9768213079999999 podStartE2EDuration="6.97574124s" totalImagesPullingTime="4.998919932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:59.973567647 +0000 UTC m=+523.862444618" watchObservedRunningTime="2026-07-28 19:43:59.97574124 +0000 UTC m=+523.864618201"
Jul 28 19:44:05.992076 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:05.992048    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6rwkl"
Jul 28 19:44:06.873322 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:06.873285    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:06.928581 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:06.928547    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:06.928726 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:06.928607    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:06.930956 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:06.930936    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s7jjx\""
Jul 28 19:44:06.965771 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:06.965726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjt8\" (UniqueName: \"kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8\") pod \"authorino-84677c95db-zjlqn\" (UID: \"bc15bc0f-89f2-4f58-b398-acae8da65b04\") " pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:07.066875 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.066843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjt8\" (UniqueName: \"kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8\") pod \"authorino-84677c95db-zjlqn\" (UID: \"bc15bc0f-89f2-4f58-b398-acae8da65b04\") " pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:07.074739 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.074708    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjt8\" (UniqueName: \"kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8\") pod \"authorino-84677c95db-zjlqn\" (UID: \"bc15bc0f-89f2-4f58-b398-acae8da65b04\") " pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:07.118859 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.118831    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:07.119064 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.119053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:07.146833 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.146811    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:44:07.185782 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.185750    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:44:07.185926 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.185820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.188240 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.188191    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 19:44:07.269605 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.269571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssft6\" (UniqueName: \"kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.269813 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.269674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.273121 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.273099    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:07.275663 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:07.275634    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc15bc0f_89f2_4f58_b398_acae8da65b04.slice/crio-e6f09371ae6075d0cbc45f0904bd4dd9a63f9dc88ce24f1b41b977b7da086b0c WatchSource:0}: Error finding container e6f09371ae6075d0cbc45f0904bd4dd9a63f9dc88ce24f1b41b977b7da086b0c: Status 404 returned error can't find the container with id e6f09371ae6075d0cbc45f0904bd4dd9a63f9dc88ce24f1b41b977b7da086b0c
Jul 28 19:44:07.370426 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.370394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.370542 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.370468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssft6\" (UniqueName: \"kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.372887 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.372865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.379048 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.379028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssft6\" (UniqueName: \"kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6\") pod \"authorino-89979c88b-gtl5v\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") " pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.497474 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.497404    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:44:07.651478 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.651345    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:44:07.654049 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:07.654021    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52f946f2_17c0_48ca_9e11_e7db9fc1a400.slice/crio-568710aad7e0997a0807168fa24166edbac524f677d8cbe2c6efb0113e971911 WatchSource:0}: Error finding container 568710aad7e0997a0807168fa24166edbac524f677d8cbe2c6efb0113e971911: Status 404 returned error can't find the container with id 568710aad7e0997a0807168fa24166edbac524f677d8cbe2c6efb0113e971911
Jul 28 19:44:07.996353 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.996328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89979c88b-gtl5v" event={"ID":"52f946f2-17c0-48ca-9e11-e7db9fc1a400","Type":"ContainerStarted","Data":"568710aad7e0997a0807168fa24166edbac524f677d8cbe2c6efb0113e971911"}
Jul 28 19:44:07.997456 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:07.997434    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84677c95db-zjlqn" event={"ID":"bc15bc0f-89f2-4f58-b398-acae8da65b04","Type":"ContainerStarted","Data":"e6f09371ae6075d0cbc45f0904bd4dd9a63f9dc88ce24f1b41b977b7da086b0c"}
Jul 28 19:44:09.003463 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.003425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84677c95db-zjlqn" event={"ID":"bc15bc0f-89f2-4f58-b398-acae8da65b04","Type":"ContainerStarted","Data":"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"}
Jul 28 19:44:09.003463 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.003435    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-84677c95db-zjlqn" podUID="bc15bc0f-89f2-4f58-b398-acae8da65b04" containerName="authorino" containerID="cri-o://a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1" gracePeriod=30
Jul 28 19:44:09.005103 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.005076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89979c88b-gtl5v" event={"ID":"52f946f2-17c0-48ca-9e11-e7db9fc1a400","Type":"ContainerStarted","Data":"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"}
Jul 28 19:44:09.018615 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.018571    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84677c95db-zjlqn" podStartSLOduration=2.3507615680000002 podStartE2EDuration="3.018558571s" totalImagesPullingTime="667.797003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:09.017258976 +0000 UTC m=+532.906135948" watchObservedRunningTime="2026-07-28 19:44:09.018558571 +0000 UTC m=+532.907435520"
Jul 28 19:44:09.032646 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.032605    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-89979c88b-gtl5v" podStartSLOduration=1.516636863 podStartE2EDuration="2.032592604s" totalImagesPullingTime="515.955741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:09.030924669 +0000 UTC m=+532.919801640" watchObservedRunningTime="2026-07-28 19:44:09.032592604 +0000 UTC m=+532.921469566"
Jul 28 19:44:09.240449 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.240426    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:09.288424 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.288330    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tjt8\" (UniqueName: \"kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8\") pod \"bc15bc0f-89f2-4f58-b398-acae8da65b04\" (UID: \"bc15bc0f-89f2-4f58-b398-acae8da65b04\") "
Jul 28 19:44:09.290371 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.290346    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8" (OuterVolumeSpecName: "kube-api-access-8tjt8") pod "bc15bc0f-89f2-4f58-b398-acae8da65b04" (UID: "bc15bc0f-89f2-4f58-b398-acae8da65b04"). InnerVolumeSpecName "kube-api-access-8tjt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:44:09.389365 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:09.389328    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tjt8\" (UniqueName: \"kubernetes.io/projected/bc15bc0f-89f2-4f58-b398-acae8da65b04-kube-api-access-8tjt8\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:44:10.016931 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.016888    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc15bc0f-89f2-4f58-b398-acae8da65b04" containerID="a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1" exitCode=0
Jul 28 19:44:10.017699 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.017674    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84677c95db-zjlqn"
Jul 28 19:44:10.017969 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.017931    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84677c95db-zjlqn" event={"ID":"bc15bc0f-89f2-4f58-b398-acae8da65b04","Type":"ContainerDied","Data":"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"}
Jul 28 19:44:10.018053 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.017999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84677c95db-zjlqn" event={"ID":"bc15bc0f-89f2-4f58-b398-acae8da65b04","Type":"ContainerDied","Data":"e6f09371ae6075d0cbc45f0904bd4dd9a63f9dc88ce24f1b41b977b7da086b0c"}
Jul 28 19:44:10.018053 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.018028    2575 scope.go:117] "RemoveContainer" containerID="a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"
Jul 28 19:44:10.029649 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.029629    2575 scope.go:117] "RemoveContainer" containerID="a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"
Jul 28 19:44:10.029983 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:44:10.029961    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1\": container with ID starting with a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1 not found: ID does not exist" containerID="a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"
Jul 28 19:44:10.030084 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.029988    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1"} err="failed to get container status \"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1\": rpc error: code = NotFound desc = could not find container \"a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1\": container with ID starting with a4ea825feb2f4159d5599c64e8c0605b37dcedb3b7d31a77a993bfffe8d4e1b1 not found: ID does not exist"
Jul 28 19:44:10.040938 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.040910    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:10.044529 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.044508    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84677c95db-zjlqn"]
Jul 28 19:44:10.697093 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:10.697053    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc15bc0f-89f2-4f58-b398-acae8da65b04" path="/var/lib/kubelet/pods/bc15bc0f-89f2-4f58-b398-acae8da65b04/volumes"
Jul 28 19:44:14.047105 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.047074    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:14.047442 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.047423    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc15bc0f-89f2-4f58-b398-acae8da65b04" containerName="authorino"
Jul 28 19:44:14.047442 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.047435    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc15bc0f-89f2-4f58-b398-acae8da65b04" containerName="authorino"
Jul 28 19:44:14.047520 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.047495    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc15bc0f-89f2-4f58-b398-acae8da65b04" containerName="authorino"
Jul 28 19:44:14.050127 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.050108    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.053257 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.053236    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:44:14.053366 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.053246    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-grlnt\""
Jul 28 19:44:14.053426 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.053404    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:44:14.062641 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.062620    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:14.133327 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.133289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vp8\" (UniqueName: \"kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.133496 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.133382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.227901 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.227866    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:44:14.230333 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.230310    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.234540 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.234511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.234651 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.234618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vp8\" (UniqueName: \"kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.234715 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:44:14.234665    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 28 19:44:14.234810 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:44:14.234787    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert podName:19a9001e-82ff-4f37-aad0-513b8eff5578 nodeName:}" failed. No retries permitted until 2026-07-28 19:44:14.734765785 +0000 UTC m=+538.623642737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert") pod "maas-controller-5f9f7b797f-bm5zs" (UID: "19a9001e-82ff-4f37-aad0-513b8eff5578") : secret "maas-controller-webhook-cert" not found
Jul 28 19:44:14.240797 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.240771    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:44:14.242593 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.242571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vp8\" (UniqueName: \"kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.336193 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.336149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njjmr\" (UniqueName: \"kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.336193 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.336196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.437363 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.437323    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njjmr\" (UniqueName: \"kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.437363 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.437363    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.439902 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.439878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.446676 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.446651    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njjmr\" (UniqueName: \"kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr\") pod \"maas-controller-54bcf7858d-jbr26\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") " pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.541517 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.541472    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:14.673458 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.673430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:44:14.676186 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:14.676154    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3335c2f2_3ce6_407f_8353_4124dfc2ca8f.slice/crio-ebe6bfd1e7b42a2d54fd7e9092d1c7ae6aeb4a77070aff3ed09c8f0e91521d29 WatchSource:0}: Error finding container ebe6bfd1e7b42a2d54fd7e9092d1c7ae6aeb4a77070aff3ed09c8f0e91521d29: Status 404 returned error can't find the container with id ebe6bfd1e7b42a2d54fd7e9092d1c7ae6aeb4a77070aff3ed09c8f0e91521d29
Jul 28 19:44:14.740953 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.740906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.743217 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.743185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") pod \"maas-controller-5f9f7b797f-bm5zs\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") " pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:14.961601 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:14.961500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:15.044948 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:15.044911    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-jbr26" event={"ID":"3335c2f2-3ce6-407f-8353-4124dfc2ca8f","Type":"ContainerStarted","Data":"ebe6bfd1e7b42a2d54fd7e9092d1c7ae6aeb4a77070aff3ed09c8f0e91521d29"}
Jul 28 19:44:15.092296 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:15.092268    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:15.095706 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:15.095679    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a9001e_82ff_4f37_aad0_513b8eff5578.slice/crio-3ae1de986cab2f16a16fd169fb2fc98d726401bfbb780325d5abd621fd578f98 WatchSource:0}: Error finding container 3ae1de986cab2f16a16fd169fb2fc98d726401bfbb780325d5abd621fd578f98: Status 404 returned error can't find the container with id 3ae1de986cab2f16a16fd169fb2fc98d726401bfbb780325d5abd621fd578f98
Jul 28 19:44:16.059993 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:16.059954    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" event={"ID":"19a9001e-82ff-4f37-aad0-513b8eff5578","Type":"ContainerStarted","Data":"3ae1de986cab2f16a16fd169fb2fc98d726401bfbb780325d5abd621fd578f98"}
Jul 28 19:44:17.265206 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:17.265182    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:44:18.071657 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.071616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-jbr26" event={"ID":"3335c2f2-3ce6-407f-8353-4124dfc2ca8f","Type":"ContainerStarted","Data":"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"}
Jul 28 19:44:18.071886 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.071690    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:18.073373 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.073346    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" event={"ID":"19a9001e-82ff-4f37-aad0-513b8eff5578","Type":"ContainerStarted","Data":"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"}
Jul 28 19:44:18.073516 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.073497    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:18.095195 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.095149    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54bcf7858d-jbr26" podStartSLOduration=1.504709995 podStartE2EDuration="4.095131335s" totalImagesPullingTime="2.59042134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:18.094473715 +0000 UTC m=+541.983350686" watchObservedRunningTime="2026-07-28 19:44:18.095131335 +0000 UTC m=+541.984008312"
Jul 28 19:44:18.115670 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:18.115620    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" podStartSLOduration=1.9499304290000001 podStartE2EDuration="4.115604867s" totalImagesPullingTime="2.165674438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:18.113795672 +0000 UTC m=+542.002672643" watchObservedRunningTime="2026-07-28 19:44:18.115604867 +0000 UTC m=+542.004481838"
Jul 28 19:44:29.080180 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.080141    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:29.080553 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.080344    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:44:29.144291 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.144257    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:29.144496 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.144474    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" podUID="19a9001e-82ff-4f37-aad0-513b8eff5578" containerName="manager" containerID="cri-o://8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856" gracePeriod=30
Jul 28 19:44:29.389823 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.389794    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:29.477002 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.476970    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7vp8\" (UniqueName: \"kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8\") pod \"19a9001e-82ff-4f37-aad0-513b8eff5578\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") "
Jul 28 19:44:29.477169 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.477058    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") pod \"19a9001e-82ff-4f37-aad0-513b8eff5578\" (UID: \"19a9001e-82ff-4f37-aad0-513b8eff5578\") "
Jul 28 19:44:29.479131 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.479100    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8" (OuterVolumeSpecName: "kube-api-access-k7vp8") pod "19a9001e-82ff-4f37-aad0-513b8eff5578" (UID: "19a9001e-82ff-4f37-aad0-513b8eff5578"). InnerVolumeSpecName "kube-api-access-k7vp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:44:29.479246 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.479154    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "19a9001e-82ff-4f37-aad0-513b8eff5578" (UID: "19a9001e-82ff-4f37-aad0-513b8eff5578"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:44:29.578481 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.578447    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7vp8\" (UniqueName: \"kubernetes.io/projected/19a9001e-82ff-4f37-aad0-513b8eff5578-kube-api-access-k7vp8\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:44:29.578481 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:29.578474    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a9001e-82ff-4f37-aad0-513b8eff5578-webhook-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:44:30.130629 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.130596    2575 generic.go:356] "Generic (PLEG): container finished" podID="19a9001e-82ff-4f37-aad0-513b8eff5578" containerID="8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856" exitCode=0
Jul 28 19:44:30.131059 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.130645    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs"
Jul 28 19:44:30.131059 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.130686    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" event={"ID":"19a9001e-82ff-4f37-aad0-513b8eff5578","Type":"ContainerDied","Data":"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"}
Jul 28 19:44:30.131059 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.130754    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9f7b797f-bm5zs" event={"ID":"19a9001e-82ff-4f37-aad0-513b8eff5578","Type":"ContainerDied","Data":"3ae1de986cab2f16a16fd169fb2fc98d726401bfbb780325d5abd621fd578f98"}
Jul 28 19:44:30.131059 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.130775    2575 scope.go:117] "RemoveContainer" containerID="8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"
Jul 28 19:44:30.140418 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.140398    2575 scope.go:117] "RemoveContainer" containerID="8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"
Jul 28 19:44:30.140701 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:44:30.140683    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856\": container with ID starting with 8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856 not found: ID does not exist" containerID="8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"
Jul 28 19:44:30.140798 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.140713    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856"} err="failed to get container status \"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856\": rpc error: code = NotFound desc = could not find container \"8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856\": container with ID starting with 8fefab40b7762bd217f48f9206c48f5984e56d28ddc6b65d6f800c5993032856 not found: ID does not exist"
Jul 28 19:44:30.152294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.152242    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:30.154671 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.154651    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f9f7b797f-bm5zs"]
Jul 28 19:44:30.696616 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:30.696586    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a9001e-82ff-4f37-aad0-513b8eff5578" path="/var/lib/kubelet/pods/19a9001e-82ff-4f37-aad0-513b8eff5578/volumes"
Jul 28 19:44:51.614232 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.614197    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"]
Jul 28 19:44:51.614748 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.614592    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a9001e-82ff-4f37-aad0-513b8eff5578" containerName="manager"
Jul 28 19:44:51.614748 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.614604    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a9001e-82ff-4f37-aad0-513b8eff5578" containerName="manager"
Jul 28 19:44:51.614748 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.614710    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a9001e-82ff-4f37-aad0-513b8eff5578" containerName="manager"
Jul 28 19:44:51.619487 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.619465    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.622027 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.622002    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 19:44:51.622178 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.622160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 19:44:51.632085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.632056    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"]
Jul 28 19:44:51.667403 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.667369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-maas-api-tls\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.667543 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.667428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-openshift-service-ca\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.667543 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.667530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxkh\" (UniqueName: \"kubernetes.io/projected/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-kube-api-access-wvxkh\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.768081 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.768044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-maas-api-tls\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.768260 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.768136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-openshift-service-ca\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.768333 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.768256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxkh\" (UniqueName: \"kubernetes.io/projected/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-kube-api-access-wvxkh\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.768980 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.768959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-openshift-service-ca\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.770526 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.770506    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-maas-api-tls\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.778212 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.778189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxkh\" (UniqueName: \"kubernetes.io/projected/077d0a51-2f26-4f99-a2de-aa355e4cd1cd-kube-api-access-wvxkh\") pod \"maas-api-5f8bcdcb76-htjrq\" (UID: \"077d0a51-2f26-4f99-a2de-aa355e4cd1cd\") " pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:51.929528 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:51.929442    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:52.077188 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.077162    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"]
Jul 28 19:44:52.079800 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:52.079772    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077d0a51_2f26_4f99_a2de_aa355e4cd1cd.slice/crio-a88b7d661d283da1f0cd390282f041f7b6f908e13cd25751f180031938d19200 WatchSource:0}: Error finding container a88b7d661d283da1f0cd390282f041f7b6f908e13cd25751f180031938d19200: Status 404 returned error can't find the container with id a88b7d661d283da1f0cd390282f041f7b6f908e13cd25751f180031938d19200
Jul 28 19:44:52.231432 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.231352    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq" event={"ID":"077d0a51-2f26-4f99-a2de-aa355e4cd1cd","Type":"ContainerStarted","Data":"a88b7d661d283da1f0cd390282f041f7b6f908e13cd25751f180031938d19200"}
Jul 28 19:44:52.284300 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.284267    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"]
Jul 28 19:44:52.287457 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.287436    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.291130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.291105    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pwjxj\""
Jul 28 19:44:52.291254 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.291170    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 19:44:52.303478 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.303450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"]
Jul 28 19:44:52.375416 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.375378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntqg\" (UniqueName: \"kubernetes.io/projected/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-kube-api-access-rntqg\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.375617 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.375514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.377851 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.377824    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qqsrk"]
Jul 28 19:44:52.380160 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.380144    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.394919 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.394892    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qqsrk"]
Jul 28 19:44:52.476409 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.476371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntqg\" (UniqueName: \"kubernetes.io/projected/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-kube-api-access-rntqg\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.476584 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.476422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed95a3f-c063-4014-be4a-b1083e72e5b0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.476584 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.476510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjtv6\" (UniqueName: \"kubernetes.io/projected/9ed95a3f-c063-4014-be4a-b1083e72e5b0-kube-api-access-bjtv6\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.476584 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.476560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.477273 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.477249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.487365 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.487315    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntqg\" (UniqueName: \"kubernetes.io/projected/2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0-kube-api-access-rntqg\") pod \"payload-pre-processing-687dbb6cf6-dfhb5\" (UID: \"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.577724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.577685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed95a3f-c063-4014-be4a-b1083e72e5b0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.577724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.577746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtv6\" (UniqueName: \"kubernetes.io/projected/9ed95a3f-c063-4014-be4a-b1083e72e5b0-kube-api-access-bjtv6\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.578368 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.578349    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed95a3f-c063-4014-be4a-b1083e72e5b0-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.586917 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.586894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjtv6\" (UniqueName: \"kubernetes.io/projected/9ed95a3f-c063-4014-be4a-b1083e72e5b0-kube-api-access-bjtv6\") pod \"payload-processing-5cc6cd444b-qqsrk\" (UID: \"9ed95a3f-c063-4014-be4a-b1083e72e5b0\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.598724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.598703    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:52.690185 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.690136    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:52.744188 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.744152    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"]
Jul 28 19:44:52.748126 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:52.748092    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2943ecc8_1f7d_460d_939d_7cf8f4ce8fc0.slice/crio-2e00b51dce542b64e66002a6497f1663f2c8219cca012f56d38fc78e2ce31b70 WatchSource:0}: Error finding container 2e00b51dce542b64e66002a6497f1663f2c8219cca012f56d38fc78e2ce31b70: Status 404 returned error can't find the container with id 2e00b51dce542b64e66002a6497f1663f2c8219cca012f56d38fc78e2ce31b70
Jul 28 19:44:52.905998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:52.905954    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qqsrk"]
Jul 28 19:44:52.920161 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:44:52.919898    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed95a3f_c063_4014_be4a_b1083e72e5b0.slice/crio-69ea230bfd31372db7d781f644148a6c444d184994600cfc773afd791b13120b WatchSource:0}: Error finding container 69ea230bfd31372db7d781f644148a6c444d184994600cfc773afd791b13120b: Status 404 returned error can't find the container with id 69ea230bfd31372db7d781f644148a6c444d184994600cfc773afd791b13120b
Jul 28 19:44:53.239217 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:53.239180    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5" event={"ID":"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0","Type":"ContainerStarted","Data":"2e00b51dce542b64e66002a6497f1663f2c8219cca012f56d38fc78e2ce31b70"}
Jul 28 19:44:53.240821 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:53.240788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk" event={"ID":"9ed95a3f-c063-4014-be4a-b1083e72e5b0","Type":"ContainerStarted","Data":"69ea230bfd31372db7d781f644148a6c444d184994600cfc773afd791b13120b"}
Jul 28 19:44:56.260014 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.259923    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5" event={"ID":"2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0","Type":"ContainerStarted","Data":"f9a285dff9f7d3d6cffda7f882613d582e825ccc2342d890bb113c3f1c449540"}
Jul 28 19:44:56.260436 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.260246    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:44:56.261584 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.261554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk" event={"ID":"9ed95a3f-c063-4014-be4a-b1083e72e5b0","Type":"ContainerStarted","Data":"557571c0631bb61d0a9087ea72adb690edacb084b4a80737716ce5ef3a238d66"}
Jul 28 19:44:56.261705 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.261651    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:44:56.263084 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.263062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq" event={"ID":"077d0a51-2f26-4f99-a2de-aa355e4cd1cd","Type":"ContainerStarted","Data":"ff330dcec1f2938798caff61ce222261c05d0be1c842aaed34366f3f91745026"}
Jul 28 19:44:56.263212 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.263198    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:44:56.279380 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.279320    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5" podStartSLOduration=1.079002984 podStartE2EDuration="4.279301153s" totalImagesPullingTime="3.200298169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:56.277337635 +0000 UTC m=+580.166214608" watchObservedRunningTime="2026-07-28 19:44:56.279301153 +0000 UTC m=+580.168178128"
Jul 28 19:44:56.300700 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.300641    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq" podStartSLOduration=2.881114016 podStartE2EDuration="5.300618287s" totalImagesPullingTime="2.419504271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:56.298500015 +0000 UTC m=+580.187376988" watchObservedRunningTime="2026-07-28 19:44:56.300618287 +0000 UTC m=+580.189495260"
Jul 28 19:44:56.316307 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:44:56.316252    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk" podStartSLOduration=1.282672172 podStartE2EDuration="4.316236196s" totalImagesPullingTime="3.033564024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:56.315985415 +0000 UTC m=+580.204862387" watchObservedRunningTime="2026-07-28 19:44:56.316236196 +0000 UTC m=+580.205113167"
Jul 28 19:45:00.142825 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.142780    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"]
Jul 28 19:45:00.145383 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.145361    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"
Jul 28 19:45:00.147871 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.147851    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 19:45:00.155524 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.155502    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"]
Jul 28 19:45:00.155665 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.155560    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"
Jul 28 19:45:00.287528 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.287503    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"]
Jul 28 19:45:00.290452 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:45:00.290426    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb20187_f48a_4dc4_bd63_e51fad11f836.slice/crio-7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980 WatchSource:0}: Error finding container 7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980: Status 404 returned error can't find the container with id 7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980
Jul 28 19:45:00.292968 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.292940    2575 generic.go:356] "Generic (PLEG): container finished" podID="efd39ec7-ad59-4d89-bab6-22dc6504b258" containerID="b868b0e4c86b8361716a6e51f5e13270655c8bbee2edbf4a3aa6d5a5a0896e11" exitCode=0
Jul 28 19:45:00.293058 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.292988    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" event={"ID":"efd39ec7-ad59-4d89-bab6-22dc6504b258","Type":"ContainerDied","Data":"b868b0e4c86b8361716a6e51f5e13270655c8bbee2edbf4a3aa6d5a5a0896e11"}
Jul 28 19:45:00.293428 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.293413    2575 scope.go:117] "RemoveContainer" containerID="b868b0e4c86b8361716a6e51f5e13270655c8bbee2edbf4a3aa6d5a5a0896e11"
Jul 28 19:45:00.294348 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.294322    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:45:00.294440 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.294383    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:45:00.294440 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:00.294417    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 19:45:01.299460 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:01.299422    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv" event={"ID":"fdb20187-f48a-4dc4-bd63-e51fad11f836","Type":"ContainerStarted","Data":"7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980"}
Jul 28 19:45:01.301297 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:01.301274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" event={"ID":"efd39ec7-ad59-4d89-bab6-22dc6504b258","Type":"ContainerStarted","Data":"86256c1f44c23d3e5868d85e76f9435cdd6fdbacfeb711836e5cc1cb1c08ca27"}
Jul 28 19:45:01.886753 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:01.886061    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:45:01.889755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:01.887059    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lcz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:45:01.889755 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:01.887114    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" podUID="efd39ec7-ad59-4d89-bab6-22dc6504b258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:45:02.291700 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:02.291618    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f8bcdcb76-htjrq"
Jul 28 19:45:02.816928 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:02.816890    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:45:02.883805 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:02.883146    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lcz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:45:02.883805 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:02.883211    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" podUID="efd39ec7-ad59-4d89-bab6-22dc6504b258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:45:03.883757 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:03.880233    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lcz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:45:03.883757 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:03.880312    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" podUID="efd39ec7-ad59-4d89-bab6-22dc6504b258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:45:04.318660 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:04.318620    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv" event={"ID":"fdb20187-f48a-4dc4-bd63-e51fad11f836","Type":"ContainerStarted","Data":"e3f4f7613ddb047b376a9a699ff9d0dac26ae264613287bbe050102fc6267b26"}
Jul 28 19:45:04.336542 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:04.336489    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv" podStartSLOduration=0.700717861 podStartE2EDuration="4.336473197s" totalImagesPullingTime="3.635755336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:04.335245733 +0000 UTC m=+588.224122705" watchObservedRunningTime="2026-07-28 19:45:04.336473197 +0000 UTC m=+588.225350168"
Jul 28 19:45:04.879083 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:04.879041    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lcz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:45:04.879293 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:04.879131    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q" podUID="efd39ec7-ad59-4d89-bab6-22dc6504b258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:45:05.324811 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:05.324772    2575 generic.go:356] "Generic (PLEG): container finished" podID="fdb20187-f48a-4dc4-bd63-e51fad11f836" containerID="e3f4f7613ddb047b376a9a699ff9d0dac26ae264613287bbe050102fc6267b26" exitCode=0
Jul 28 19:45:05.325271 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:05.324846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv" event={"ID":"fdb20187-f48a-4dc4-bd63-e51fad11f836","Type":"ContainerDied","Data":"e3f4f7613ddb047b376a9a699ff9d0dac26ae264613287bbe050102fc6267b26"}
Jul 28 19:45:05.882053 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:05.882019    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:45:05.882998 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:05.882977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lcz4q"
Jul 28 19:45:06.452291 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:06.452268    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"
Jul 28 19:45:07.272620 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:07.272589    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dfhb5"
Jul 28 19:45:07.272806 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:07.272664    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-qqsrk"
Jul 28 19:45:07.336702 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:07.336670    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"
Jul 28 19:45:07.336702 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:07.336692    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv" event={"ID":"fdb20187-f48a-4dc4-bd63-e51fad11f836","Type":"ContainerDied","Data":"7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980"}
Jul 28 19:45:07.336910 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:07.336753    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d633d8d08c9e68e0863cf26de457f6d26033a8e75766c64e0bf1cc9b7bc6980"
Jul 28 19:45:12.059524 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.059480    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"]
Jul 28 19:45:12.059961 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.059943    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb20187-f48a-4dc4-bd63-e51fad11f836" containerName="cleanup"
Jul 28 19:45:12.060008 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.059964    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb20187-f48a-4dc4-bd63-e51fad11f836" containerName="cleanup"
Jul 28 19:45:12.060055 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.060045    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb20187-f48a-4dc4-bd63-e51fad11f836" containerName="cleanup"
Jul 28 19:45:12.063491 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.063471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.065888 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.065866    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:45:12.066062 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.066037    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jpg4f\""
Jul 28 19:45:12.066706 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.066682    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:45:12.066840 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.066819    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 19:45:12.075940 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.075912    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"]
Jul 28 19:45:12.195842 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.196022 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.196022 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.196022 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqm8g\" (UniqueName: \"kubernetes.io/projected/3262c41b-b5dc-4ce1-9b87-0afca7193c12-kube-api-access-cqm8g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.196022 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.196022 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.195942    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.296892 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.296857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.296901    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.296924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.296942    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqm8g\" (UniqueName: \"kubernetes.io/projected/3262c41b-b5dc-4ce1-9b87-0afca7193c12-kube-api-access-cqm8g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297085 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.297066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297303 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.297106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297404 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.297382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297463 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.297392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.297463 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.297442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.299193 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.299171    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3262c41b-b5dc-4ce1-9b87-0afca7193c12-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.299432 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.299413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3262c41b-b5dc-4ce1-9b87-0afca7193c12-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.304569 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.304546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqm8g\" (UniqueName: \"kubernetes.io/projected/3262c41b-b5dc-4ce1-9b87-0afca7193c12-kube-api-access-cqm8g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j\" (UID: \"3262c41b-b5dc-4ce1-9b87-0afca7193c12\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.376215 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.376180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:12.516325 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:12.516300    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"]
Jul 28 19:45:12.518646 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:45:12.518618    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3262c41b_b5dc_4ce1_9b87_0afca7193c12.slice/crio-5a0aff8c8ad24dfd929958ee63ec61583b71227825b91c7c167f7ebea535233a WatchSource:0}: Error finding container 5a0aff8c8ad24dfd929958ee63ec61583b71227825b91c7c167f7ebea535233a: Status 404 returned error can't find the container with id 5a0aff8c8ad24dfd929958ee63ec61583b71227825b91c7c167f7ebea535233a
Jul 28 19:45:13.368435 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:13.368395    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j" event={"ID":"3262c41b-b5dc-4ce1-9b87-0afca7193c12","Type":"ContainerStarted","Data":"5a0aff8c8ad24dfd929958ee63ec61583b71227825b91c7c167f7ebea535233a"}
Jul 28 19:45:14.374418 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:14.374384    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j" event={"ID":"3262c41b-b5dc-4ce1-9b87-0afca7193c12","Type":"ContainerStarted","Data":"bae8539d3fe5f11057c0b9d17ba281ee11390a183ad8684191aaa3cda9026479"}
Jul 28 19:45:14.390570 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:14.390513    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j" podStartSLOduration=0.910579414 podStartE2EDuration="2.390494855s" totalImagesPullingTime="1.479915441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:14.39034377 +0000 UTC m=+598.279220740" watchObservedRunningTime="2026-07-28 19:45:14.390494855 +0000 UTC m=+598.279371825"
Jul 28 19:45:16.525655 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:16.525622    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:45:16.533294 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:16.533273    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:45:22.377276 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:22.377223    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:22.377692 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:22.377300    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:22.389770 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:22.389724    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:22.423717 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:22.423685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j"
Jul 28 19:45:42.750954 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.750880    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"]
Jul 28 19:45:42.754277 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.754259    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.756621 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.756595    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 19:45:42.769074 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.769041    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"]
Jul 28 19:45:42.883824 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.883790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.883994 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.883842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.883994 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.883864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.883994 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.883940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzf4r\" (UniqueName: \"kubernetes.io/projected/3b3b3ed7-feb7-420d-b5a4-6defa375e139-kube-api-access-zzf4r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.883994 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.883987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.884124 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.884009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985093 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985285 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf4r\" (UniqueName: \"kubernetes.io/projected/3b3b3ed7-feb7-420d-b5a4-6defa375e139-kube-api-access-zzf4r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985507 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985308    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985668 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985819 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.985819 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.985770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.987471 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.987445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3b3ed7-feb7-420d-b5a4-6defa375e139-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.987591 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.987575    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3b3ed7-feb7-420d-b5a4-6defa375e139-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:42.992617 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:42.992599    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzf4r\" (UniqueName: \"kubernetes.io/projected/3b3b3ed7-feb7-420d-b5a4-6defa375e139-kube-api-access-zzf4r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2jxmm\" (UID: \"3b3b3ed7-feb7-420d-b5a4-6defa375e139\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:43.070093 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:43.070018    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:43.202534 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:43.202505    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"]
Jul 28 19:45:43.204776 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:45:43.204749    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3b3ed7_feb7_420d_b5a4_6defa375e139.slice/crio-1a4f57417eb360e3f693308f923ee9b1599c7e90abbd378bee3ffbbc999b22af WatchSource:0}: Error finding container 1a4f57417eb360e3f693308f923ee9b1599c7e90abbd378bee3ffbbc999b22af: Status 404 returned error can't find the container with id 1a4f57417eb360e3f693308f923ee9b1599c7e90abbd378bee3ffbbc999b22af
Jul 28 19:45:43.206458 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:43.206439    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:45:43.517254 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:43.517225    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm" event={"ID":"3b3b3ed7-feb7-420d-b5a4-6defa375e139","Type":"ContainerStarted","Data":"1a4f57417eb360e3f693308f923ee9b1599c7e90abbd378bee3ffbbc999b22af"}
Jul 28 19:45:44.525529 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:44.525487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm" event={"ID":"3b3b3ed7-feb7-420d-b5a4-6defa375e139","Type":"ContainerStarted","Data":"27e8604a1d4855d5271c16de4199df216d427d005db10372c27d85e413e04afc"}
Jul 28 19:45:44.551290 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:44.551228    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm" podStartSLOduration=2.243315703 podStartE2EDuration="2.551210498s" totalImagesPullingTime="307.894795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:44.549951455 +0000 UTC m=+628.438828419" watchObservedRunningTime="2026-07-28 19:45:44.551210498 +0000 UTC m=+628.440087468"
Jul 28 19:45:53.070772 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:53.070718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:53.071248 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:53.070811    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:53.083003 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:53.082975    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:45:53.578020 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:45:53.577992    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2jxmm"
Jul 28 19:47:04.975615 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:04.975575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:47:04.976465 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:04.976347    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-89979c88b-gtl5v" podUID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" containerName="authorino" containerID="cri-o://811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556" gracePeriod=30
Jul 28 19:47:05.225118 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.225094    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:47:05.338056 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.338023    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssft6\" (UniqueName: \"kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6\") pod \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") "
Jul 28 19:47:05.338221 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.338115    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert\") pod \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\" (UID: \"52f946f2-17c0-48ca-9e11-e7db9fc1a400\") "
Jul 28 19:47:05.340074 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.340046    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6" (OuterVolumeSpecName: "kube-api-access-ssft6") pod "52f946f2-17c0-48ca-9e11-e7db9fc1a400" (UID: "52f946f2-17c0-48ca-9e11-e7db9fc1a400"). InnerVolumeSpecName "kube-api-access-ssft6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:47:05.348724 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.348697    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "52f946f2-17c0-48ca-9e11-e7db9fc1a400" (UID: "52f946f2-17c0-48ca-9e11-e7db9fc1a400"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:47:05.439318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.439280    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f946f2-17c0-48ca-9e11-e7db9fc1a400-tls-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:47:05.439318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.439310    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssft6\" (UniqueName: \"kubernetes.io/projected/52f946f2-17c0-48ca-9e11-e7db9fc1a400-kube-api-access-ssft6\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:47:05.907602 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.907560    2575 generic.go:356] "Generic (PLEG): container finished" podID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" containerID="811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556" exitCode=0
Jul 28 19:47:05.907826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.907606    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89979c88b-gtl5v"
Jul 28 19:47:05.907826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.907653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89979c88b-gtl5v" event={"ID":"52f946f2-17c0-48ca-9e11-e7db9fc1a400","Type":"ContainerDied","Data":"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"}
Jul 28 19:47:05.907826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.907696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89979c88b-gtl5v" event={"ID":"52f946f2-17c0-48ca-9e11-e7db9fc1a400","Type":"ContainerDied","Data":"568710aad7e0997a0807168fa24166edbac524f677d8cbe2c6efb0113e971911"}
Jul 28 19:47:05.907826 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.907716    2575 scope.go:117] "RemoveContainer" containerID="811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"
Jul 28 19:47:05.918897 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.918876    2575 scope.go:117] "RemoveContainer" containerID="811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"
Jul 28 19:47:05.919150 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:47:05.919130    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556\": container with ID starting with 811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556 not found: ID does not exist" containerID="811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"
Jul 28 19:47:05.919193 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.919159    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556"} err="failed to get container status \"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556\": rpc error: code = NotFound desc = could not find container \"811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556\": container with ID starting with 811f97e3713f8de0866ca625912ec0e56f5f558735e75427ed584256ba272556 not found: ID does not exist"
Jul 28 19:47:05.931506 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.931484    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:47:05.935045 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:05.935023    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-89979c88b-gtl5v"]
Jul 28 19:47:06.695440 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:06.695408    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" path="/var/lib/kubelet/pods/52f946f2-17c0-48ca-9e11-e7db9fc1a400/volumes"
Jul 28 19:47:51.407163 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.407119    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:47:51.407787 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.407315    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54bcf7858d-jbr26" podUID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" containerName="manager" containerID="cri-o://cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20" gracePeriod=30
Jul 28 19:47:51.664396 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.664336    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:47:51.753851 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.753816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert\") pod \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") "
Jul 28 19:47:51.754020 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.753896    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njjmr\" (UniqueName: \"kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr\") pod \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\" (UID: \"3335c2f2-3ce6-407f-8353-4124dfc2ca8f\") "
Jul 28 19:47:51.755812 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.755784    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3335c2f2-3ce6-407f-8353-4124dfc2ca8f" (UID: "3335c2f2-3ce6-407f-8353-4124dfc2ca8f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:47:51.755926 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.755910    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr" (OuterVolumeSpecName: "kube-api-access-njjmr") pod "3335c2f2-3ce6-407f-8353-4124dfc2ca8f" (UID: "3335c2f2-3ce6-407f-8353-4124dfc2ca8f"). InnerVolumeSpecName "kube-api-access-njjmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:47:51.854772 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.854713    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njjmr\" (UniqueName: \"kubernetes.io/projected/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-kube-api-access-njjmr\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:47:51.854772 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:51.854768    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3335c2f2-3ce6-407f-8353-4124dfc2ca8f-webhook-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:47:52.132993 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.132957    2575 generic.go:356] "Generic (PLEG): container finished" podID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" containerID="cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20" exitCode=0
Jul 28 19:47:52.133171 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.132999    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-jbr26"
Jul 28 19:47:52.133171 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.133036    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-jbr26" event={"ID":"3335c2f2-3ce6-407f-8353-4124dfc2ca8f","Type":"ContainerDied","Data":"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"}
Jul 28 19:47:52.133171 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.133083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-jbr26" event={"ID":"3335c2f2-3ce6-407f-8353-4124dfc2ca8f","Type":"ContainerDied","Data":"ebe6bfd1e7b42a2d54fd7e9092d1c7ae6aeb4a77070aff3ed09c8f0e91521d29"}
Jul 28 19:47:52.133171 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.133106    2575 scope.go:117] "RemoveContainer" containerID="cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"
Jul 28 19:47:52.142100 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.142082    2575 scope.go:117] "RemoveContainer" containerID="cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"
Jul 28 19:47:52.142362 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:47:52.142347    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20\": container with ID starting with cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20 not found: ID does not exist" containerID="cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"
Jul 28 19:47:52.142404 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.142370    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20"} err="failed to get container status \"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20\": rpc error: code = NotFound desc = could not find container \"cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20\": container with ID starting with cabb2218b93d9c30897877ca1bd2045c6fe55e8ac3f1a2b3a42878596bfc9a20 not found: ID does not exist"
Jul 28 19:47:52.154060 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.154034    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:47:52.157106 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.157082    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-jbr26"]
Jul 28 19:47:52.696329 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:47:52.696295    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" path="/var/lib/kubelet/pods/3335c2f2-3ce6-407f-8353-4124dfc2ca8f/volumes"
Jul 28 19:48:52.991650 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.991570    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.991964    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" containerName="authorino"
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.991976    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" containerName="authorino"
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.991988    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" containerName="manager"
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.991994    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" containerName="manager"
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.992057    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f946f2-17c0-48ca-9e11-e7db9fc1a400" containerName="authorino"
Jul 28 19:48:52.992130 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.992067    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3335c2f2-3ce6-407f-8353-4124dfc2ca8f" containerName="manager"
Jul 28 19:48:52.995235 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.995219    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:52.998126 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.998107    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:48:52.998853 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.998837    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:48:52.998969 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:52.998932    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-grlnt\""
Jul 28 19:48:53.004504 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.004480    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:48:53.110509 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.110478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.110669 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.110560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kmwn\" (UniqueName: \"kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.211860 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.211825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kmwn\" (UniqueName: \"kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.212024 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.211876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.214339 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.214310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.220188 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.220162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kmwn\" (UniqueName: \"kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn\") pod \"maas-controller-54bcf7858d-vch24\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") " pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.307105 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.307018    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:53.441318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:53.441287    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:48:53.444630 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:48:53.444604    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff742469_13d9_4ac1_8cb7_09521328f2d9.slice/crio-0bc8ba90b7466b5cb5e60895f8df3b78a55c08ca12759a0853dc91788235117d WatchSource:0}: Error finding container 0bc8ba90b7466b5cb5e60895f8df3b78a55c08ca12759a0853dc91788235117d: Status 404 returned error can't find the container with id 0bc8ba90b7466b5cb5e60895f8df3b78a55c08ca12759a0853dc91788235117d
Jul 28 19:48:54.454760 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:54.454711    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-vch24" event={"ID":"ff742469-13d9-4ac1-8cb7-09521328f2d9","Type":"ContainerStarted","Data":"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"}
Jul 28 19:48:54.455230 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:54.454772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-vch24" event={"ID":"ff742469-13d9-4ac1-8cb7-09521328f2d9","Type":"ContainerStarted","Data":"0bc8ba90b7466b5cb5e60895f8df3b78a55c08ca12759a0853dc91788235117d"}
Jul 28 19:48:54.455230 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:54.454840    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:48:54.477796 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:48:54.477711    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54bcf7858d-vch24" podStartSLOduration=2.042502013 podStartE2EDuration="2.477697299s" totalImagesPullingTime="435.195286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:54.476287089 +0000 UTC m=+818.365164061" watchObservedRunningTime="2026-07-28 19:48:54.477697299 +0000 UTC m=+818.366574269"
Jul 28 19:49:05.462586 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:05.462556    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:49:10.160910 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.160875    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:49:10.161368 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.161093    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54bcf7858d-vch24" podUID="ff742469-13d9-4ac1-8cb7-09521328f2d9" containerName="manager" containerID="cri-o://cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5" gracePeriod=30
Jul 28 19:49:10.407563 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.407537    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:49:10.474791 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.474692    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert\") pod \"ff742469-13d9-4ac1-8cb7-09521328f2d9\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") "
Jul 28 19:49:10.474791 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.474754    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kmwn\" (UniqueName: \"kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn\") pod \"ff742469-13d9-4ac1-8cb7-09521328f2d9\" (UID: \"ff742469-13d9-4ac1-8cb7-09521328f2d9\") "
Jul 28 19:49:10.476787 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.476713    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ff742469-13d9-4ac1-8cb7-09521328f2d9" (UID: "ff742469-13d9-4ac1-8cb7-09521328f2d9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:49:10.476787 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.476766    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn" (OuterVolumeSpecName: "kube-api-access-6kmwn") pod "ff742469-13d9-4ac1-8cb7-09521328f2d9" (UID: "ff742469-13d9-4ac1-8cb7-09521328f2d9"). InnerVolumeSpecName "kube-api-access-6kmwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:49:10.532178 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.532148    2575 generic.go:356] "Generic (PLEG): container finished" podID="ff742469-13d9-4ac1-8cb7-09521328f2d9" containerID="cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5" exitCode=0
Jul 28 19:49:10.532318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.532186    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-vch24"
Jul 28 19:49:10.532318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.532230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-vch24" event={"ID":"ff742469-13d9-4ac1-8cb7-09521328f2d9","Type":"ContainerDied","Data":"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"}
Jul 28 19:49:10.532318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.532282    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-vch24" event={"ID":"ff742469-13d9-4ac1-8cb7-09521328f2d9","Type":"ContainerDied","Data":"0bc8ba90b7466b5cb5e60895f8df3b78a55c08ca12759a0853dc91788235117d"}
Jul 28 19:49:10.532318 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.532306    2575 scope.go:117] "RemoveContainer" containerID="cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"
Jul 28 19:49:10.541642 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.541622    2575 scope.go:117] "RemoveContainer" containerID="cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"
Jul 28 19:49:10.541925 ip-10-0-130-70 kubenswrapper[2575]: E0728 19:49:10.541905    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5\": container with ID starting with cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5 not found: ID does not exist" containerID="cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"
Jul 28 19:49:10.541970 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.541934    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5"} err="failed to get container status \"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5\": rpc error: code = NotFound desc = could not find container \"cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5\": container with ID starting with cdce4bfbd452ad2958c578d958a54fb2f2d50878abd27b7f6a1b040b6867b6c5 not found: ID does not exist"
Jul 28 19:49:10.554427 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.554401    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:49:10.558427 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.558407    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-vch24"]
Jul 28 19:49:10.576108 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.576086    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff742469-13d9-4ac1-8cb7-09521328f2d9-webhook-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:10.576108 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.576104    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kmwn\" (UniqueName: \"kubernetes.io/projected/ff742469-13d9-4ac1-8cb7-09521328f2d9-kube-api-access-6kmwn\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:10.695574 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:10.695540    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff742469-13d9-4ac1-8cb7-09521328f2d9" path="/var/lib/kubelet/pods/ff742469-13d9-4ac1-8cb7-09521328f2d9/volumes"
Jul 28 19:49:11.394034 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.393998    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54bcf7858d-kv7nq"]
Jul 28 19:49:11.394410 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.394382    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff742469-13d9-4ac1-8cb7-09521328f2d9" containerName="manager"
Jul 28 19:49:11.394410 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.394393    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff742469-13d9-4ac1-8cb7-09521328f2d9" containerName="manager"
Jul 28 19:49:11.394477 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.394457    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff742469-13d9-4ac1-8cb7-09521328f2d9" containerName="manager"
Jul 28 19:49:11.397538 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.397515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.399873 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.399851    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:49:11.400422 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.400406    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:49:11.400500 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.400408    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-grlnt\""
Jul 28 19:49:11.406291 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.406270    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-kv7nq"]
Jul 28 19:49:11.483823 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.483783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwvm2\" (UniqueName: \"kubernetes.io/projected/6eb12912-cfde-4981-8ff2-d35874285a88-kube-api-access-gwvm2\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.483823 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.483825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6eb12912-cfde-4981-8ff2-d35874285a88-webhook-cert\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.585115 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.585082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwvm2\" (UniqueName: \"kubernetes.io/projected/6eb12912-cfde-4981-8ff2-d35874285a88-kube-api-access-gwvm2\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.585115 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.585117    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6eb12912-cfde-4981-8ff2-d35874285a88-webhook-cert\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.587508 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.587489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6eb12912-cfde-4981-8ff2-d35874285a88-webhook-cert\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.594258 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.594231    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwvm2\" (UniqueName: \"kubernetes.io/projected/6eb12912-cfde-4981-8ff2-d35874285a88-kube-api-access-gwvm2\") pod \"maas-controller-54bcf7858d-kv7nq\" (UID: \"6eb12912-cfde-4981-8ff2-d35874285a88\") " pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.709751 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.709650    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:11.843429 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:11.843400    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54bcf7858d-kv7nq"]
Jul 28 19:49:11.845583 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:49:11.845553    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb12912_cfde_4981_8ff2_d35874285a88.slice/crio-92bc06cffb43c9dbfb97e5173abf2f85095ce38677af208309e09b3ae85c3653 WatchSource:0}: Error finding container 92bc06cffb43c9dbfb97e5173abf2f85095ce38677af208309e09b3ae85c3653: Status 404 returned error can't find the container with id 92bc06cffb43c9dbfb97e5173abf2f85095ce38677af208309e09b3ae85c3653
Jul 28 19:49:12.546072 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:12.545984    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-kv7nq" event={"ID":"6eb12912-cfde-4981-8ff2-d35874285a88","Type":"ContainerStarted","Data":"957e0ae1f1ceacf70136eab879eb5f1c1bc74f0ceef6c28361f94b738852f257"}
Jul 28 19:49:12.546072 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:12.546044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54bcf7858d-kv7nq" event={"ID":"6eb12912-cfde-4981-8ff2-d35874285a88","Type":"ContainerStarted","Data":"92bc06cffb43c9dbfb97e5173abf2f85095ce38677af208309e09b3ae85c3653"}
Jul 28 19:49:12.546499 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:12.546079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:49:12.564556 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:12.564510    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54bcf7858d-kv7nq" podStartSLOduration=1.232380105 podStartE2EDuration="1.564494285s" totalImagesPullingTime="332.11418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:49:12.562394153 +0000 UTC m=+836.451271125" watchObservedRunningTime="2026-07-28 19:49:12.564494285 +0000 UTC m=+836.453371248"
Jul 28 19:49:23.554434 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:49:23.554404    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54bcf7858d-kv7nq"
Jul 28 19:50:16.561086 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:50:16.561057    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:50:16.567563 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:50:16.567543    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:55:15.319952 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.319917    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"]
Jul 28 19:55:15.323349 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.323333    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.326995 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.326975    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lcnz8\""
Jul 28 19:55:15.353604 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.353581    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"]
Jul 28 19:55:15.441871 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.441838    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgfst\" (UniqueName: \"kubernetes.io/projected/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-kube-api-access-rgfst\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.442037 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.441876    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.543002 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.542971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgfst\" (UniqueName: \"kubernetes.io/projected/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-kube-api-access-rgfst\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.543002 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.543007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.543419 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.543403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.552220 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.552202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgfst\" (UniqueName: \"kubernetes.io/projected/99f01c69-5d89-41fb-ac7b-8bbb37ebef69-kube-api-access-rgfst\") pod \"kuadrant-operator-controller-manager-86878c769f-g9bbb\" (UID: \"99f01c69-5d89-41fb-ac7b-8bbb37ebef69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.634469 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.634446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:15.771143 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:15.771111    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"]
Jul 28 19:55:15.774375 ip-10-0-130-70 kubenswrapper[2575]: W0728 19:55:15.774347    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f01c69_5d89_41fb_ac7b_8bbb37ebef69.slice/crio-cce090d6877bcefb83a60913acc8d803ca188d9f36d9e04d8152b19d75db2368 WatchSource:0}: Error finding container cce090d6877bcefb83a60913acc8d803ca188d9f36d9e04d8152b19d75db2368: Status 404 returned error can't find the container with id cce090d6877bcefb83a60913acc8d803ca188d9f36d9e04d8152b19d75db2368
Jul 28 19:55:16.301485 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.301447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb" event={"ID":"99f01c69-5d89-41fb-ac7b-8bbb37ebef69","Type":"ContainerStarted","Data":"4078fb8ad41f3efad9f2b22e9fad7664d0731e919df246623cd6a7257fd9fa2e"}
Jul 28 19:55:16.301654 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.301494    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb" event={"ID":"99f01c69-5d89-41fb-ac7b-8bbb37ebef69","Type":"ContainerStarted","Data":"cce090d6877bcefb83a60913acc8d803ca188d9f36d9e04d8152b19d75db2368"}
Jul 28 19:55:16.301654 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.301532    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 19:55:16.323237 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.323192    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb" podStartSLOduration=1.32317634 podStartE2EDuration="1.32317634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:16.322018006 +0000 UTC m=+1200.210894979" watchObservedRunningTime="2026-07-28 19:55:16.32317634 +0000 UTC m=+1200.212053313"
Jul 28 19:55:16.592859 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.592832    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:55:16.596833 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:16.596813    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 19:55:27.309284 ip-10-0-130-70 kubenswrapper[2575]: I0728 19:55:27.309212    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g9bbb"
Jul 28 20:00:16.624080 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:00:16.624053    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:00:16.626555 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:00:16.625177    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:02:46.398361 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.398232    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:02:46.402372 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.402349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.404776 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.404756    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fc68dda7-openshift-default-dockercfg-jzhbr\""
Jul 28 20:02:46.424906 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.424884    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:02:46.448478 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ztz\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448889 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448889 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448681    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.448889 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.448783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.549847 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.549817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550036 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.549873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550036 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.549921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550036 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.549942    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550112    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ztz\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550143    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550409 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550409 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550409 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550556 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550504    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550611 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.550823 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.550807    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.552240 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.552218    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.552447 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.552429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.558639 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.558618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.558991 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.558968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ztz\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz\") pod \"e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") " pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.712665 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.712574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:46.852446 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.852417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:02:46.854323 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:02:46.854299    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba694374_5275_4cf0_8a16_364a968ad991.slice/crio-0e161be58f908b87b5cbb176c793f02a676e0e31887ec4aa2d155cdfa3371e48 WatchSource:0}: Error finding container 0e161be58f908b87b5cbb176c793f02a676e0e31887ec4aa2d155cdfa3371e48: Status 404 returned error can't find the container with id 0e161be58f908b87b5cbb176c793f02a676e0e31887ec4aa2d155cdfa3371e48
Jul 28 20:02:46.856518 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.856483    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:02:46.856600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.856549    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:02:46.856600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:46.856576    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:02:47.556153 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:47.556115    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerStarted","Data":"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"}
Jul 28 20:02:47.556528 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:47.556179    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerStarted","Data":"0e161be58f908b87b5cbb176c793f02a676e0e31887ec4aa2d155cdfa3371e48"}
Jul 28 20:02:47.577411 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:47.577353    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podStartSLOduration=1.5773314200000002 podStartE2EDuration="1.57733142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:47.576386245 +0000 UTC m=+1651.465263221" watchObservedRunningTime="2026-07-28 20:02:47.57733142 +0000 UTC m=+1651.466208395"
Jul 28 20:02:47.713031 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:47.712997    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:47.717882 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:47.717847    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:48.560788 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:48.560753    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:48.561724 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:48.561704    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:02:49.063573 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.063535    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:02:49.066619 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.066596    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.069776 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.069755    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fc68dda7-serving-cert\""
Jul 28 20:02:49.073676 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.073656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.073803 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.073783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dzj\" (UniqueName: \"kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.073870 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.073812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.093680 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.093629    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:02:49.174953 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.174912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.175132 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.174979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dzj\" (UniqueName: \"kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.175132 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.175007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.175132 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:02:49.175064    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-serving-cert: secret "maas-api-e2e-ait-fc68dda7-serving-cert" not found
Jul 28 20:02:49.175132 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:02:49.175133    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls podName:9cda7464-dfc7-4c74-a28b-6fa3d79df9c6 nodeName:}" failed. No retries permitted until 2026-07-28 20:02:49.675117179 +0000 UTC m=+1653.563994128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls") pod "maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" (UID: "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6") : secret "maas-api-e2e-ait-fc68dda7-serving-cert" not found
Jul 28 20:02:49.175644 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.175627    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.193146 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.193115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dzj\" (UniqueName: \"kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.680456 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.680420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.683518 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.683495    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") pod \"maas-api-e2e-ait-fc68dda7-7689556666-tcsvb\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.801592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.801561    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:02:49.805056 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.805037    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.807305 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.807287    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fc68dda7-dockercfg-5p7rz\""
Jul 28 20:02:49.808167 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.807964    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fc68dda7\""
Jul 28 20:02:49.818032 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.818007    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:02:49.881879 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.881845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ms8v\" (UniqueName: \"kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.882050 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.881906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.978742 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.978638    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:49.982569 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.982540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ms8v\" (UniqueName: \"kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.982698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.982622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.983362 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.983336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:49.990545 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:49.990499    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ms8v\" (UniqueName: \"kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v\") pod \"payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:50.119210 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.119178    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:50.131674 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.131650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:02:50.133972 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:02:50.133945    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cda7464_dfc7_4c74_a28b_6fa3d79df9c6.slice/crio-142141cf05469c4b39a00235bc3f1bbf1f9cfc6c22e0b2167e67cc0d23a47c5a WatchSource:0}: Error finding container 142141cf05469c4b39a00235bc3f1bbf1f9cfc6c22e0b2167e67cc0d23a47c5a: Status 404 returned error can't find the container with id 142141cf05469c4b39a00235bc3f1bbf1f9cfc6c22e0b2167e67cc0d23a47c5a
Jul 28 20:02:50.135904 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.135883    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:02:50.257938 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.257918    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:02:50.260808 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:02:50.260770    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73b112a_d986_441b_854d_7eb0e1bd3afd.slice/crio-ed128faf24f212f46396d04c1f7e5e941cf20973fce4a17d2d6de9cd10030ed8 WatchSource:0}: Error finding container ed128faf24f212f46396d04c1f7e5e941cf20973fce4a17d2d6de9cd10030ed8: Status 404 returned error can't find the container with id ed128faf24f212f46396d04c1f7e5e941cf20973fce4a17d2d6de9cd10030ed8
Jul 28 20:02:50.572911 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.572822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" event={"ID":"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6","Type":"ContainerStarted","Data":"142141cf05469c4b39a00235bc3f1bbf1f9cfc6c22e0b2167e67cc0d23a47c5a"}
Jul 28 20:02:50.574622 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.574591    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" event={"ID":"a73b112a-d986-441b-854d-7eb0e1bd3afd","Type":"ContainerStarted","Data":"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"}
Jul 28 20:02:50.574765 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.574647    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" event={"ID":"a73b112a-d986-441b-854d-7eb0e1bd3afd","Type":"ContainerStarted","Data":"ed128faf24f212f46396d04c1f7e5e941cf20973fce4a17d2d6de9cd10030ed8"}
Jul 28 20:02:50.594824 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:50.594766    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" podStartSLOduration=1.594746766 podStartE2EDuration="1.594746766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:50.593170327 +0000 UTC m=+1654.482047298" watchObservedRunningTime="2026-07-28 20:02:50.594746766 +0000 UTC m=+1654.483623734"
Jul 28 20:02:51.582101 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:51.582063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" event={"ID":"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6","Type":"ContainerStarted","Data":"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"}
Jul 28 20:02:51.582549 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:51.582133    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:02:51.582549 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:51.582511    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:02:51.599342 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:51.599290    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" podStartSLOduration=2.011531859 podStartE2EDuration="2.599275351s" totalImagesPullingTime="587.743492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:02:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:51.596602699 +0000 UTC m=+1655.485479669" watchObservedRunningTime="2026-07-28 20:02:51.599275351 +0000 UTC m=+1655.488152322"
Jul 28 20:02:57.595442 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:02:57.595410    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:03:02.589821 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:02.589789    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:03:04.339053 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.338475    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:39202->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Jul 28 20:03:04.339053 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.338545    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:39202->10.134.0.45:15021: read: connection reset by peer"
Jul 28 20:03:04.653434 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.653354    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba694374-5275-4cf0-8a16-364a968ad991" containerID="9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786" exitCode=0
Jul 28 20:03:04.653577 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.653426    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerDied","Data":"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"}
Jul 28 20:03:04.653871 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.653857    2575 scope.go:117] "RemoveContainer" containerID="9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"
Jul 28 20:03:04.654890 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.654861    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:04.654956 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.654929    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:04.654995 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:04.654963    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:05.660649 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:05.660616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerStarted","Data":"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"}
Jul 28 20:03:05.713724 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:05.713686    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:05.713927 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:05.713905    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 20:03:05.713970 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:05.713954    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 20:03:06.713515 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:06.713481    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 20:03:06.713930 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:06.713532    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 20:03:07.714024 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:07.713992    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 20:03:07.714411 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:07.714044    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 20:03:08.713519 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:08.713486    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 20:03:08.713694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:08.713540    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 20:03:09.716863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:09.716831    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:09.717240 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:09.717105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:09.717810 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:09.717793    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:19.775081 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:19.775044    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:03:19.775667 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:19.775325    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" podUID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" containerName="maas-api" containerID="cri-o://d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b" gracePeriod=30
Jul 28 20:03:19.837599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:19.837565    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:03:19.837866 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:19.837816    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" podUID="a73b112a-d986-441b-854d-7eb0e1bd3afd" containerName="payload-pre-processing" containerID="cri-o://3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888" gracePeriod=30
Jul 28 20:03:20.108541 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.108494    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:03:20.109608 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.109551    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" containerID="cri-o://bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017" gracePeriod=30
Jul 28 20:03:20.123866 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.123843    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:03:20.142632 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.142607    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:03:20.193609 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.193578    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") pod \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") "
Jul 28 20:03:20.193766 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.193638    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ms8v\" (UniqueName: \"kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v\") pod \"a73b112a-d986-441b-854d-7eb0e1bd3afd\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") "
Jul 28 20:03:20.193832 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.193768    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca\") pod \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") "
Jul 28 20:03:20.193832 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.193795    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8dzj\" (UniqueName: \"kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj\") pod \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\" (UID: \"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6\") "
Jul 28 20:03:20.193832 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.193813    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume\") pod \"a73b112a-d986-441b-854d-7eb0e1bd3afd\" (UID: \"a73b112a-d986-441b-854d-7eb0e1bd3afd\") "
Jul 28 20:03:20.194185 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.194150    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" (UID: "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:20.194292 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.194246    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a73b112a-d986-441b-854d-7eb0e1bd3afd" (UID: "a73b112a-d986-441b-854d-7eb0e1bd3afd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:20.195799 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.195769    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v" (OuterVolumeSpecName: "kube-api-access-5ms8v") pod "a73b112a-d986-441b-854d-7eb0e1bd3afd" (UID: "a73b112a-d986-441b-854d-7eb0e1bd3afd"). InnerVolumeSpecName "kube-api-access-5ms8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:20.195978 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.195955    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj" (OuterVolumeSpecName: "kube-api-access-l8dzj") pod "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" (UID: "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6"). InnerVolumeSpecName "kube-api-access-l8dzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:20.196056 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.196004    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" (UID: "9cda7464-dfc7-4c74-a28b-6fa3d79df9c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:03:20.294603 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.294574    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-openshift-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.294603 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.294600    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8dzj\" (UniqueName: \"kubernetes.io/projected/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-kube-api-access-l8dzj\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.294603 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.294611    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73b112a-d986-441b-854d-7eb0e1bd3afd-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.294872 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.294621    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6-maas-api-tls\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.294872 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.294630    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ms8v\" (UniqueName: \"kubernetes.io/projected/a73b112a-d986-441b-854d-7eb0e1bd3afd-kube-api-access-5ms8v\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.745831 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.745795    2575 generic.go:356] "Generic (PLEG): container finished" podID="a73b112a-d986-441b-854d-7eb0e1bd3afd" containerID="3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888" exitCode=0
Jul 28 20:03:20.745981 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.745863    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"
Jul 28 20:03:20.745981 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.745881    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" event={"ID":"a73b112a-d986-441b-854d-7eb0e1bd3afd","Type":"ContainerDied","Data":"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"}
Jul 28 20:03:20.745981 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.745936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz" event={"ID":"a73b112a-d986-441b-854d-7eb0e1bd3afd","Type":"ContainerDied","Data":"ed128faf24f212f46396d04c1f7e5e941cf20973fce4a17d2d6de9cd10030ed8"}
Jul 28 20:03:20.745981 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.745957    2575 scope.go:117] "RemoveContainer" containerID="3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"
Jul 28 20:03:20.747508 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.747485    2575 generic.go:356] "Generic (PLEG): container finished" podID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" containerID="d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b" exitCode=0
Jul 28 20:03:20.747614 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.747520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" event={"ID":"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6","Type":"ContainerDied","Data":"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"}
Jul 28 20:03:20.747614 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.747547    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"
Jul 28 20:03:20.747614 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.747559    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb" event={"ID":"9cda7464-dfc7-4c74-a28b-6fa3d79df9c6","Type":"ContainerDied","Data":"142141cf05469c4b39a00235bc3f1bbf1f9cfc6c22e0b2167e67cc0d23a47c5a"}
Jul 28 20:03:20.755235 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.755218    2575 scope.go:117] "RemoveContainer" containerID="3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"
Jul 28 20:03:20.755504 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:03:20.755485    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888\": container with ID starting with 3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888 not found: ID does not exist" containerID="3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"
Jul 28 20:03:20.755553 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.755512    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888"} err="failed to get container status \"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888\": rpc error: code = NotFound desc = could not find container \"3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888\": container with ID starting with 3a490de32e569acf7a4d182b6e97f2081e131881b4bc9de04a9be0641c9ca888 not found: ID does not exist"
Jul 28 20:03:20.755553 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.755534    2575 scope.go:117] "RemoveContainer" containerID="d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"
Jul 28 20:03:20.763111 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.763090    2575 scope.go:117] "RemoveContainer" containerID="d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"
Jul 28 20:03:20.763355 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:03:20.763338    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b\": container with ID starting with d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b not found: ID does not exist" containerID="d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"
Jul 28 20:03:20.763409 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.763361    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b"} err="failed to get container status \"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b\": rpc error: code = NotFound desc = could not find container \"d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b\": container with ID starting with d5f3a82b739f425122fc2e216e51230e316c7dbb46e62a754448fe7509cd1e9b not found: ID does not exist"
Jul 28 20:03:20.767911 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.767888    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:03:20.776530 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.776507    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fc68dda7-7449967b56-vqgkz"]
Jul 28 20:03:20.789376 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.789352    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:03:20.793258 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:20.793239    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc68dda7-7689556666-tcsvb"]
Jul 28 20:03:22.696446 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:22.696415    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" path="/var/lib/kubelet/pods/9cda7464-dfc7-4c74-a28b-6fa3d79df9c6/volumes"
Jul 28 20:03:22.696823 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:22.696808    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73b112a-d986-441b-854d-7eb0e1bd3afd" path="/var/lib/kubelet/pods/a73b112a-d986-441b-854d-7eb0e1bd3afd/volumes"
Jul 28 20:03:24.719675 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:24.719641    2575 patch_prober.go:28] interesting pod/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 20:03:24.720037 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:24.719703    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 20:03:25.385915 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.385852    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:25.434670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434639    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.434862 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434699    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.434862 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434744    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.434862 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434764    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.434862 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434788    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.435094 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.435094 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434954    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.435094 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.434987    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.435094 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435051    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:03:25.435094 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435058    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:03:25.435367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435227    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:03:25.435367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435265    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8ztz\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz\") pod \"ba694374-5275-4cf0-8a16-364a968ad991\" (UID: \"ba694374-5275-4cf0-8a16-364a968ad991\") "
Jul 28 20:03:25.435477 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:25.435608 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435586    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.435683 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435613    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.435683 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435628    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba694374-5275-4cf0-8a16-364a968ad991-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.435683 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435643    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.435898 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.435865    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:03:25.436945 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.436922    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:03:25.437241 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.437219    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:25.437241 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.437229    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:03:25.437394 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.437376    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz" (OuterVolumeSpecName: "kube-api-access-z8ztz") pod "ba694374-5275-4cf0-8a16-364a968ad991" (UID: "ba694374-5275-4cf0-8a16-364a968ad991"). InnerVolumeSpecName "kube-api-access-z8ztz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:25.543004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.537281    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8ztz\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-kube-api-access-z8ztz\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.543004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.537316    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba694374-5275-4cf0-8a16-364a968ad991-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.543004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.537332    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba694374-5275-4cf0-8a16-364a968ad991-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.543004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.537389    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.543004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.537420    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba694374-5275-4cf0-8a16-364a968ad991-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:03:25.776869 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.776782    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba694374-5275-4cf0-8a16-364a968ad991" containerID="bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017" exitCode=0
Jul 28 20:03:25.776869 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.776845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerDied","Data":"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"}
Jul 28 20:03:25.776869 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.776849    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"
Jul 28 20:03:25.777423 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.776886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867" event={"ID":"ba694374-5275-4cf0-8a16-364a968ad991","Type":"ContainerDied","Data":"0e161be58f908b87b5cbb176c793f02a676e0e31887ec4aa2d155cdfa3371e48"}
Jul 28 20:03:25.777423 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.776908    2575 scope.go:117] "RemoveContainer" containerID="bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"
Jul 28 20:03:25.786603 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.786585    2575 scope.go:117] "RemoveContainer" containerID="9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"
Jul 28 20:03:25.794911 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.794895    2575 scope.go:117] "RemoveContainer" containerID="bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"
Jul 28 20:03:25.795166 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:03:25.795147    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017\": container with ID starting with bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017 not found: ID does not exist" containerID="bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"
Jul 28 20:03:25.795217 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.795174    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017"} err="failed to get container status \"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017\": rpc error: code = NotFound desc = could not find container \"bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017\": container with ID starting with bcb197a5d5ea463549f63f6e9f936661d4cf0ca13de80727276dc5d390107017 not found: ID does not exist"
Jul 28 20:03:25.795217 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.795191    2575 scope.go:117] "RemoveContainer" containerID="9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"
Jul 28 20:03:25.795407 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:03:25.795390    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786\": container with ID starting with 9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786 not found: ID does not exist" containerID="9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"
Jul 28 20:03:25.795448 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.795412    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786"} err="failed to get container status \"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786\": rpc error: code = NotFound desc = could not find container \"9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786\": container with ID starting with 9ce9aebb72a8865d0fc17fa1dfcc72653757bf8d2258f3f0f302a0a568a99786 not found: ID does not exist"
Jul 28 20:03:25.799235 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.799213    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:03:25.803355 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:25.803335    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fc68dda7-openshift-default-d9f654bbb-7z867"]
Jul 28 20:03:26.696456 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:26.696425    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba694374-5275-4cf0-8a16-364a968ad991" path="/var/lib/kubelet/pods/ba694374-5275-4cf0-8a16-364a968ad991/volumes"
Jul 28 20:03:55.912227 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912151    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912597    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912612    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912648    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73b112a-d986-441b-854d-7eb0e1bd3afd" containerName="payload-pre-processing"
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912653    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73b112a-d986-441b-854d-7eb0e1bd3afd" containerName="payload-pre-processing"
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912664    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:03:55.912670 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912670    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:03:55.912901 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912678    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" containerName="maas-api"
Jul 28 20:03:55.912901 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912687    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" containerName="maas-api"
Jul 28 20:03:55.912901 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912757    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:03:55.912901 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912764    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cda7464-dfc7-4c74-a28b-6fa3d79df9c6" containerName="maas-api"
Jul 28 20:03:55.912901 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.912774    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73b112a-d986-441b-854d-7eb0e1bd3afd" containerName="payload-pre-processing"
Jul 28 20:03:55.916026 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.916004    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:55.918297 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.918278    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8b9a5873-openshift-default-dockercfg-ppdk5\""
Jul 28 20:03:55.927916 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:55.927892    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:03:56.020102 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020268 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020268 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020268 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020170    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020268 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020500 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vxrr\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020500 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020500 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.020500 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.020443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.121955 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.121920    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.121955 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.121956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.121974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.121991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vxrr\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122556 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122401    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122556 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122477    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122633 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.122633 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.122604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.123087 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.123064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.124346 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.124323    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.124592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.124572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.131427 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.131403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.139135 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.139112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vxrr\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr\") pod \"e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") " pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.228068 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.227992    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:56.364476 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.364402    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:03:56.365865 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:03:56.365837    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87922d2_eac1_41c8_807e_9057a4e013ad.slice/crio-1f4bb5252b620fc18eebeacf7af9fd62fbe2dbdfb61d8392e31c5f75a1cf9407 WatchSource:0}: Error finding container 1f4bb5252b620fc18eebeacf7af9fd62fbe2dbdfb61d8392e31c5f75a1cf9407: Status 404 returned error can't find the container with id 1f4bb5252b620fc18eebeacf7af9fd62fbe2dbdfb61d8392e31c5f75a1cf9407
Jul 28 20:03:56.368285 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.368244    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:56.368379 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.368326    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:56.368379 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.368371    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:03:56.938970 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.938936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerStarted","Data":"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"}
Jul 28 20:03:56.939350 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.938991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerStarted","Data":"1f4bb5252b620fc18eebeacf7af9fd62fbe2dbdfb61d8392e31c5f75a1cf9407"}
Jul 28 20:03:56.958153 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:56.958102    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podStartSLOduration=1.958083526 podStartE2EDuration="1.958083526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:03:56.957043725 +0000 UTC m=+1720.845920696" watchObservedRunningTime="2026-07-28 20:03:56.958083526 +0000 UTC m=+1720.846960498"
Jul 28 20:03:57.228486 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:57.228401    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:57.233578 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:57.233553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:57.943750 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:57.943710    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:03:57.944653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:03:57.944632    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:12.945740 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:12.945693    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:12.946133 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:12.945787    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:13.025657 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.025626    2575 generic.go:356] "Generic (PLEG): container finished" podID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerID="36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d" exitCode=0
Jul 28 20:04:13.025847 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.025682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerDied","Data":"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"}
Jul 28 20:04:13.026077 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.026059    2575 scope.go:117] "RemoveContainer" containerID="36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"
Jul 28 20:04:13.027096 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.027067    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:04:13.027151 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.027138    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:04:13.027189 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:13.027170    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:04:14.032573 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:14.032535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerStarted","Data":"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"}
Jul 28 20:04:14.228496 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:14.228459    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:14.228763 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:14.228723    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:14.228829 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:14.228785    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:15.229018 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:15.228982    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:15.229488 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:15.229057    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:16.229342 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:16.229308    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:16.229722 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:16.229361    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:17.228610 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:17.228571    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:17.228810 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:17.228632    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:18.228886 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:18.228849    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 28 20:04:18.229300 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:18.228903    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 28 20:04:19.232918 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:19.232886    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:19.233295 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:19.233265    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:19.234008 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:19.233989    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:30.515201 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:30.515168    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:04:30.515767 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:30.515387    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" containerID="cri-o://92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3" gracePeriod=30
Jul 28 20:04:34.236282 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:34.236234    2575 patch_prober.go:28] interesting pod/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 20:04:34.236653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:34.236294    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 20:04:35.781426 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.781358    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:35.898073 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898033    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898136    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898167    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vxrr\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898215    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898253    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898319    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898350    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898375    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs\") pod \"e87922d2-eac1-41c8-807e-9057a4e013ad\" (UID: \"e87922d2-eac1-41c8-807e-9057a4e013ad\") "
Jul 28 20:04:35.898700 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898566    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:04:35.898802 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898781    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:35.898873 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898824    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:04:35.898873 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:35.899017 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.898993    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:04:35.899665 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.899638    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:04:35.900510 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.900479    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:35.900611 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.900599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:04:35.900692 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.900675    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr" (OuterVolumeSpecName: "kube-api-access-5vxrr") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "kube-api-access-5vxrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:35.900811 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.900791    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e87922d2-eac1-41c8-807e-9057a4e013ad" (UID: "e87922d2-eac1-41c8-807e-9057a4e013ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:04:36.000003 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:35.999966    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000003 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000006    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000021    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e87922d2-eac1-41c8-807e-9057a4e013ad-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000035    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000049    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e87922d2-eac1-41c8-807e-9057a4e013ad-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000064    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000075    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e87922d2-eac1-41c8-807e-9057a4e013ad-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.000254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.000088    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vxrr\" (UniqueName: \"kubernetes.io/projected/e87922d2-eac1-41c8-807e-9057a4e013ad-kube-api-access-5vxrr\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:04:36.148161 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.148123    2575 generic.go:356] "Generic (PLEG): container finished" podID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerID="92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3" exitCode=0
Jul 28 20:04:36.148323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.148172    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"
Jul 28 20:04:36.148323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.148210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerDied","Data":"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"}
Jul 28 20:04:36.148323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.148265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57" event={"ID":"e87922d2-eac1-41c8-807e-9057a4e013ad","Type":"ContainerDied","Data":"1f4bb5252b620fc18eebeacf7af9fd62fbe2dbdfb61d8392e31c5f75a1cf9407"}
Jul 28 20:04:36.148323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.148287    2575 scope.go:117] "RemoveContainer" containerID="92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"
Jul 28 20:04:36.158324 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.158303    2575 scope.go:117] "RemoveContainer" containerID="36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"
Jul 28 20:04:36.166796 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.166775    2575 scope.go:117] "RemoveContainer" containerID="92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"
Jul 28 20:04:36.167083 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:04:36.167058    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3\": container with ID starting with 92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3 not found: ID does not exist" containerID="92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"
Jul 28 20:04:36.167204 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.167087    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3"} err="failed to get container status \"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3\": rpc error: code = NotFound desc = could not find container \"92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3\": container with ID starting with 92184525ca41cb1d422e8541cb7d5125aa6c39855f2ff5715ee2a7b3097397f3 not found: ID does not exist"
Jul 28 20:04:36.167204 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.167105    2575 scope.go:117] "RemoveContainer" containerID="36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"
Jul 28 20:04:36.167347 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:04:36.167329    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d\": container with ID starting with 36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d not found: ID does not exist" containerID="36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"
Jul 28 20:04:36.167409 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.167350    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d"} err="failed to get container status \"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d\": rpc error: code = NotFound desc = could not find container \"36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d\": container with ID starting with 36b7a649b1b713528f256e7d22884ae9a040585033a675462352e00651ef880d not found: ID does not exist"
Jul 28 20:04:36.172297 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.172249    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:04:36.174635 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.174612    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8b9a5873-openshift-default-7d6cc4445d-qhd57"]
Jul 28 20:04:36.697239 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:04:36.697198    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" path="/var/lib/kubelet/pods/e87922d2-eac1-41c8-807e-9057a4e013ad/volumes"
Jul 28 20:05:16.653711 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:05:16.653674    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:05:16.654677 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:05:16.654655    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:06:43.694929 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.694844    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695284    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695297    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695311    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695317    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695376    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:06:43.695387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.695388    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba694374-5275-4cf0-8a16-364a968ad991" containerName="istio-proxy"
Jul 28 20:06:43.698595 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.698577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.701662 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.701641    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9667e4b-openshift-default-dockercfg-p9s5m\""
Jul 28 20:06:43.717659 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.717635    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:06:43.850196 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx49c\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850508 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850508 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850508 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.850604 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.850524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952031 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.951947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952031 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.951991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952050    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx49c\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952088    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952274 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952711 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952711 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952458    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952711 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.952711 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.952601    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.953036 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.953014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.954433 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.954408    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.954938 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.954916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.959416 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.959391    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx49c\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:43.959559 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:43.959393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token\") pod \"e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") " pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:44.009231 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.009199    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:44.354246 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.354211    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:06:44.355484 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:06:44.355455    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13021067_94ee_40a1_8e58_2e1d62d03931.slice/crio-d871d398688971e0c2a1ac3f2464a2a2eb8fd9c1417e2c7584ebf1c80dc20e3c WatchSource:0}: Error finding container d871d398688971e0c2a1ac3f2464a2a2eb8fd9c1417e2c7584ebf1c80dc20e3c: Status 404 returned error can't find the container with id d871d398688971e0c2a1ac3f2464a2a2eb8fd9c1417e2c7584ebf1c80dc20e3c
Jul 28 20:06:44.357503 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.357471    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:06:44.357600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.357548    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:06:44.357600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.357592    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:06:44.778353 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.778257    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerStarted","Data":"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"}
Jul 28 20:06:44.778353 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.778320    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerStarted","Data":"d871d398688971e0c2a1ac3f2464a2a2eb8fd9c1417e2c7584ebf1c80dc20e3c"}
Jul 28 20:06:44.799000 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:44.798953    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podStartSLOduration=1.798937616 podStartE2EDuration="1.798937616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:44.796512143 +0000 UTC m=+1888.685389114" watchObservedRunningTime="2026-07-28 20:06:44.798937616 +0000 UTC m=+1888.687814619"
Jul 28 20:06:45.010188 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:45.010156    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:45.015144 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:45.015122    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:45.790270 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:45.790236    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:06:45.791435 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:06:45.791408    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:00.792017 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.791981    2575 patch_prober.go:28] interesting pod/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 28 20:07:00.792447 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.792048    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 28 20:07:00.871131 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.871094    2575 generic.go:356] "Generic (PLEG): container finished" podID="13021067-94ee-40a1-8e58-2e1d62d03931" containerID="b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3" exitCode=0
Jul 28 20:07:00.871332 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.871166    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerDied","Data":"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"}
Jul 28 20:07:00.871596 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.871572    2575 scope.go:117] "RemoveContainer" containerID="b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"
Jul 28 20:07:00.872608 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.872576    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:00.872677 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.872653    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:00.872714 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:00.872690    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:01.885055 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:01.885011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerStarted","Data":"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"}
Jul 28 20:07:02.009704 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:02.009670    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:02.010046 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:02.010014    2575 patch_prober.go:28] interesting pod/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 28 20:07:02.010155 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:02.010081    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 28 20:07:03.009895 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:03.009860    2575 patch_prober.go:28] interesting pod/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 28 20:07:03.010279 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:03.009914    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 28 20:07:04.010301 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:04.010264    2575 patch_prober.go:28] interesting pod/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 28 20:07:04.010691 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:04.010323    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 28 20:07:05.010639 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:05.010605    2575 patch_prober.go:28] interesting pod/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 28 20:07:05.011044 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:05.010663    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 28 20:07:06.013880 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:06.013849    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:06.014335 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:06.014234    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:06.014845 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:06.014818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:36.566516 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:36.566482    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:07:36.567256 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:36.567223    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy" containerID="cri-o://9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3" gracePeriod=30
Jul 28 20:07:41.821103 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:41.821080    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:42.001232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001152    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx49c\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001261    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001359    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001382    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001404    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001592 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001504    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token\") pod \"13021067-94ee-40a1-8e58-2e1d62d03931\" (UID: \"13021067-94ee-40a1-8e58-2e1d62d03931\") "
Jul 28 20:07:42.001838 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:07:42.001838 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001644    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:07:42.001838 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001699    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:07:42.002006 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001892    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.002006 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001915    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.002006 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001932    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.002006 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.001969    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:07:42.002320 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.002301    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data" (OuterVolumeSpecName: "istio-data") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:07:42.003920 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.003873    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:07:42.004029 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.003941    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:07:42.004029 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.003998    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token" (OuterVolumeSpecName: "istio-token") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:07:42.004108 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.004068    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c" (OuterVolumeSpecName: "kube-api-access-xx49c") pod "13021067-94ee-40a1-8e58-2e1d62d03931" (UID: "13021067-94ee-40a1-8e58-2e1d62d03931"). InnerVolumeSpecName "kube-api-access-xx49c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:07:42.091800 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.091767    2575 generic.go:356] "Generic (PLEG): container finished" podID="13021067-94ee-40a1-8e58-2e1d62d03931" containerID="9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3" exitCode=0
Jul 28 20:07:42.091948 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.091832    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"
Jul 28 20:07:42.091948 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.091860    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerDied","Data":"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"}
Jul 28 20:07:42.091948 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.091923    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk" event={"ID":"13021067-94ee-40a1-8e58-2e1d62d03931","Type":"ContainerDied","Data":"d871d398688971e0c2a1ac3f2464a2a2eb8fd9c1417e2c7584ebf1c80dc20e3c"}
Jul 28 20:07:42.092118 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.091957    2575 scope.go:117] "RemoveContainer" containerID="9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"
Jul 28 20:07:42.102864 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.102817    2575 scope.go:117] "RemoveContainer" containerID="b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"
Jul 28 20:07:42.103255 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103235    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13021067-94ee-40a1-8e58-2e1d62d03931-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.103321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103261    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.103321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103275    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13021067-94ee-40a1-8e58-2e1d62d03931-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.103321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103288    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13021067-94ee-40a1-8e58-2e1d62d03931-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.103321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103299    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.103321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.103311    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx49c\" (UniqueName: \"kubernetes.io/projected/13021067-94ee-40a1-8e58-2e1d62d03931-kube-api-access-xx49c\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:07:42.110923 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.110900    2575 scope.go:117] "RemoveContainer" containerID="9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"
Jul 28 20:07:42.111153 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:07:42.111138    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3\": container with ID starting with 9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3 not found: ID does not exist" containerID="9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"
Jul 28 20:07:42.111199 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.111160    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3"} err="failed to get container status \"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3\": rpc error: code = NotFound desc = could not find container \"9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3\": container with ID starting with 9232cf4bd96cba938fc7664872d1f4b0047299e354e01ca5bb2bf9adf51ed5b3 not found: ID does not exist"
Jul 28 20:07:42.111199 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.111174    2575 scope.go:117] "RemoveContainer" containerID="b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"
Jul 28 20:07:42.111405 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:07:42.111388    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3\": container with ID starting with b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3 not found: ID does not exist" containerID="b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"
Jul 28 20:07:42.111446 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.111413    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3"} err="failed to get container status \"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3\": rpc error: code = NotFound desc = could not find container \"b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3\": container with ID starting with b97b705a24a752719045dc4532ad6d8afcab523d30e4a4c73f21e87cf381b2d3 not found: ID does not exist"
Jul 28 20:07:42.115887 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.115868    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:07:42.120082 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.120062    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9667e4b-openshift-default-cfd8cb8d8-j6lgk"]
Jul 28 20:07:42.695702 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:42.695662    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" path="/var/lib/kubelet/pods/13021067-94ee-40a1-8e58-2e1d62d03931/volumes"
Jul 28 20:07:56.549647 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.549613    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550389    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550414    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550460    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550470    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550580    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:07:56.551021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.550596    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e87922d2-eac1-41c8-807e-9057a4e013ad" containerName="istio-proxy"
Jul 28 20:07:56.553922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.553901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.556239 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.556220    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-27915281-openshift-default-dockercfg-gc2gr\""
Jul 28 20:07:56.564926 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.564893    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:07:56.618701 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.618701 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.618893 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qxfn\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.618893 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.618893 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.618893 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.619024 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.619024 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.619024 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.618986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719545 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719602    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxfn\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.719694 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720047 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719762    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720047 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720047 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719886    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720047 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.719997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720267 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.720183    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720327 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.720292    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.720469 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.720451    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.721814 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.721791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.722213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.722194    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.727045 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.727026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.727238 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.727221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxfn\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn\") pod \"e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") " pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:56.864282 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:56.864244    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:57.004676 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.004598    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:07:57.008553 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:07:57.008528    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe596f63_658a_4e11_85ea_8573a2513ee8.slice/crio-8c5c3fb45485497e998baa4724ba488c35afb9ad6ae131943d21a1c875d4e4e8 WatchSource:0}: Error finding container 8c5c3fb45485497e998baa4724ba488c35afb9ad6ae131943d21a1c875d4e4e8: Status 404 returned error can't find the container with id 8c5c3fb45485497e998baa4724ba488c35afb9ad6ae131943d21a1c875d4e4e8
Jul 28 20:07:57.011252 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.011214    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:57.011328 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.011300    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:57.011368 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.011341    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:07:57.168013 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.167972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerStarted","Data":"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"}
Jul 28 20:07:57.168185 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.168064    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerStarted","Data":"8c5c3fb45485497e998baa4724ba488c35afb9ad6ae131943d21a1c875d4e4e8"}
Jul 28 20:07:57.187440 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.187378    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podStartSLOduration=1.187359152 podStartE2EDuration="1.187359152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:57.186851188 +0000 UTC m=+1961.075728173" watchObservedRunningTime="2026-07-28 20:07:57.187359152 +0000 UTC m=+1961.076236125"
Jul 28 20:07:57.865232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:57.865190    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:58.869895 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:58.869864    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:59.179754 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:59.179639    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:07:59.180604 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:07:59.180585    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:08:14.181606 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.181565    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 28 20:08:14.182036 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.181630    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 28 20:08:14.258774 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.258741    2575 generic.go:356] "Generic (PLEG): container finished" podID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerID="7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7" exitCode=0
Jul 28 20:08:14.258943 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.258802    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerDied","Data":"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"}
Jul 28 20:08:14.259227 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.259209    2575 scope.go:117] "RemoveContainer" containerID="7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"
Jul 28 20:08:14.260267 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.260243    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:08:14.260323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.260305    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:08:14.260359 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:14.260339    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:08:15.265778 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:15.265749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerStarted","Data":"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"}
Jul 28 20:08:15.865271 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:15.865230    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:08:15.865801 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:15.865775    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 28 20:08:15.865991 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:15.865967    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 28 20:08:16.865202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:16.865163    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 28 20:08:16.865836 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:16.865799    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 28 20:08:17.865631 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:17.865586    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 28 20:08:17.866148 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:17.865669    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 28 20:08:18.865597 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:18.865551    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 28 20:08:18.865836 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:18.865619    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 28 20:08:19.868559 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:19.868525    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:08:19.869015 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:19.868930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:08:19.869464 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:08:19.869445    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:10:16.683974 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:16.683945    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:10:16.686661 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:16.686624    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:10:30.440076 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:30.440043    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:10:30.440595 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:30.440251    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" containerID="cri-o://a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580" gracePeriod=30
Jul 28 20:10:34.871272 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:34.871229    2575 patch_prober.go:28] interesting pod/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 20:10:34.871721 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:34.871302    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 20:10:35.696848 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.696823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:10:35.801616 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801525    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801616 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801572    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qxfn\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801616 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801713    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801770    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801801    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801826    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.801922 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.801867    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:35.802231 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802071    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy\") pod \"fe596f63-658a-4e11-85ea-8573a2513ee8\" (UID: \"fe596f63-658a-4e11-85ea-8573a2513ee8\") "
Jul 28 20:10:35.802231 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802083    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:35.802231 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802107    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:35.802403 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:35.802745 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802702    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.802858 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802757    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.802858 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802776    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe596f63-658a-4e11-85ea-8573a2513ee8-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.802858 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.802792    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.803620 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.803593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:35.804266 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.804239    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:35.804349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.804328    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:10:35.804349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.804334    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn" (OuterVolumeSpecName: "kube-api-access-7qxfn") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "kube-api-access-7qxfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:35.804453 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.804432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe596f63-658a-4e11-85ea-8573a2513ee8" (UID: "fe596f63-658a-4e11-85ea-8573a2513ee8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:35.903628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.903589    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.903628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.903623    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qxfn\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-kube-api-access-7qxfn\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.903628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.903637    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.904115 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.903645    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.904115 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.903653    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe596f63-658a-4e11-85ea-8573a2513ee8-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:10:35.985585 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.985543    2575 generic.go:356] "Generic (PLEG): container finished" podID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerID="a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580" exitCode=0
Jul 28 20:10:35.985863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.985597    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"
Jul 28 20:10:35.985863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.985622    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerDied","Data":"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"}
Jul 28 20:10:35.985863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.985683    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj" event={"ID":"fe596f63-658a-4e11-85ea-8573a2513ee8","Type":"ContainerDied","Data":"8c5c3fb45485497e998baa4724ba488c35afb9ad6ae131943d21a1c875d4e4e8"}
Jul 28 20:10:35.985863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.985706    2575 scope.go:117] "RemoveContainer" containerID="a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"
Jul 28 20:10:35.995378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:35.995360    2575 scope.go:117] "RemoveContainer" containerID="7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"
Jul 28 20:10:36.003680 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.003658    2575 scope.go:117] "RemoveContainer" containerID="a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"
Jul 28 20:10:36.004019 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:10:36.003994    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580\": container with ID starting with a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580 not found: ID does not exist" containerID="a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"
Jul 28 20:10:36.004109 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.004025    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580"} err="failed to get container status \"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580\": rpc error: code = NotFound desc = could not find container \"a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580\": container with ID starting with a44fe47d50d789340697616b493721ce24ec945992b0874c94fbb9fbd9c60580 not found: ID does not exist"
Jul 28 20:10:36.004109 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.004043    2575 scope.go:117] "RemoveContainer" containerID="7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"
Jul 28 20:10:36.004320 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:10:36.004301    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7\": container with ID starting with 7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7 not found: ID does not exist" containerID="7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"
Jul 28 20:10:36.004362 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.004328    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7"} err="failed to get container status \"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7\": rpc error: code = NotFound desc = could not find container \"7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7\": container with ID starting with 7e152f36e34b12637264c15e3ab1c1dd99d629343bdf1a1225b50421c934f2b7 not found: ID does not exist"
Jul 28 20:10:36.009022 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.008997    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:10:36.014724 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.014700    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-27915281-openshift-default-7ffd9f8948-6kpsj"]
Jul 28 20:10:36.695850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:10:36.695809    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" path="/var/lib/kubelet/pods/fe596f63-658a-4e11-85ea-8573a2513ee8/volumes"
Jul 28 20:11:54.892714 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.892674    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:11:54.893320 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893303    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893385 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893324    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893385 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893352    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893385 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893361    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893491 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893470    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893491 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893483    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe596f63-658a-4e11-85ea-8573a2513ee8" containerName="istio-proxy"
Jul 28 20:11:54.893566 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.893497    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="13021067-94ee-40a1-8e58-2e1d62d03931" containerName="istio-proxy"
Jul 28 20:11:54.897018 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.896998    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:54.899265 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.899237    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ba3d8d-openshift-default-dockercfg-jx7g2\""
Jul 28 20:11:54.906207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:54.906182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:11:55.023391 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023357    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b59k\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023824 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.023824 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.023620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.124906 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.124877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.124910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.124937    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.124966    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.124991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125017    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b59k\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125090 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125435 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125117    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125501 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125556 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125672 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125770 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.125919 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.125888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.127372 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.127352    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.127605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.127586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.131985 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.131962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.132293 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.132273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b59k\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k\") pod \"e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") " pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.208982 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.208910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:55.349959 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.349911    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:11:55.352512 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:11:55.352485    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3c52b0_6e87_46f5_be04_dbd0583c27fb.slice/crio-82a13b8c59dd08c4d3afa1dfe82cfa485c366822a13e9f47fcb51f75e76e2f4c WatchSource:0}: Error finding container 82a13b8c59dd08c4d3afa1dfe82cfa485c366822a13e9f47fcb51f75e76e2f4c: Status 404 returned error can't find the container with id 82a13b8c59dd08c4d3afa1dfe82cfa485c366822a13e9f47fcb51f75e76e2f4c
Jul 28 20:11:55.355173 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.355131    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:11:55.355255 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.355219    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:11:55.355295 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.355257    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:11:55.374912 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:55.374865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerStarted","Data":"82a13b8c59dd08c4d3afa1dfe82cfa485c366822a13e9f47fcb51f75e76e2f4c"}
Jul 28 20:11:56.382565 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:56.382536    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerStarted","Data":"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"}
Jul 28 20:11:56.402809 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:56.402763    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podStartSLOduration=2.402749006 podStartE2EDuration="2.402749006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:56.400448007 +0000 UTC m=+2200.289324978" watchObservedRunningTime="2026-07-28 20:11:56.402749006 +0000 UTC m=+2200.291626049"
Jul 28 20:11:57.209408 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:57.209375    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:57.214215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:57.214192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:57.387085 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:57.387053    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:57.388050 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:57.388031    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:11:59.410093 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.410058    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:11:59.413719 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.413700    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.416131 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.416110    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ba3d8d\""
Jul 28 20:11:59.416309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.416289    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ba3d8d-dockercfg-bfp9z\""
Jul 28 20:11:59.424536 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.424512    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:11:59.563665 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.563626    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn6xz\" (UniqueName: \"kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.563894 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.563701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.664858 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.664764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.665029 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.664902    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn6xz\" (UniqueName: \"kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.665511 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.665487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.681411 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.681377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn6xz\" (UniqueName: \"kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz\") pod \"payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.725579 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.725548    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:11:59.865164 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:11:59.865133    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:11:59.868398 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:11:59.868370    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db0504f_55f6_4dc4_ba80_c10e493a82fe.slice/crio-8aaa432a115ab4ead532723593a5a113727f89c9dc0f2b882de0f8087c6e336f WatchSource:0}: Error finding container 8aaa432a115ab4ead532723593a5a113727f89c9dc0f2b882de0f8087c6e336f: Status 404 returned error can't find the container with id 8aaa432a115ab4ead532723593a5a113727f89c9dc0f2b882de0f8087c6e336f
Jul 28 20:12:00.404466 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:00.404430    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" event={"ID":"8db0504f-55f6-4dc4-ba80-c10e493a82fe","Type":"ContainerStarted","Data":"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"}
Jul 28 20:12:00.404663 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:00.404495    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:12:00.404663 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:00.404519    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" event={"ID":"8db0504f-55f6-4dc4-ba80-c10e493a82fe","Type":"ContainerStarted","Data":"8aaa432a115ab4ead532723593a5a113727f89c9dc0f2b882de0f8087c6e336f"}
Jul 28 20:12:00.435438 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:00.435387    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" podStartSLOduration=1.435370553 podStartE2EDuration="1.435370553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:12:00.435041444 +0000 UTC m=+2204.323918414" watchObservedRunningTime="2026-07-28 20:12:00.435370553 +0000 UTC m=+2204.324247542"
Jul 28 20:12:11.411836 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:11.411804    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:12:13.387474 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:13.387438    2575 patch_prober.go:28] interesting pod/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 20:12:13.387885 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:13.387491    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:12:14.480082 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.480044    2575 generic.go:356] "Generic (PLEG): container finished" podID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerID="13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81" exitCode=0
Jul 28 20:12:14.480621 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.480122    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerDied","Data":"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"}
Jul 28 20:12:14.480712 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.480692    2575 scope.go:117] "RemoveContainer" containerID="13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"
Jul 28 20:12:14.481970 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.481940    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:12:14.482042 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.482014    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:12:14.482079 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:14.482049    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:12:15.491195 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:15.491140    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerStarted","Data":"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"}
Jul 28 20:12:16.209544 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:16.209510    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:12:16.209825 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:16.209806    2575 patch_prober.go:28] interesting pod/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 28 20:12:16.209886 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:16.209849    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 28 20:12:17.209678 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:17.209642    2575 patch_prober.go:28] interesting pod/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 28 20:12:17.210080 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:17.209702    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 28 20:12:18.210015 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:18.209980    2575 patch_prober.go:28] interesting pod/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 28 20:12:18.210392 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:18.210035    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 28 20:12:19.209523 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:19.209481    2575 patch_prober.go:28] interesting pod/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 28 20:12:19.209709 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:19.209555    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 28 20:12:20.213109 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:20.213075    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:12:20.213573 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:20.213469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:12:20.214053 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:12:20.214030    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:14:33.359681 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.359575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:14:33.360426 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.359975    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" podUID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" containerName="payload-pre-processing" containerID="cri-o://65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92" gracePeriod=30
Jul 28 20:14:33.633182 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.633156    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:14:33.700795 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.700762    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn6xz\" (UniqueName: \"kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz\") pod \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") "
Jul 28 20:14:33.700985 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.700887    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume\") pod \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\" (UID: \"8db0504f-55f6-4dc4-ba80-c10e493a82fe\") "
Jul 28 20:14:33.701265 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.701242    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8db0504f-55f6-4dc4-ba80-c10e493a82fe" (UID: "8db0504f-55f6-4dc4-ba80-c10e493a82fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:33.702854 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.702834    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz" (OuterVolumeSpecName: "kube-api-access-qn6xz") pod "8db0504f-55f6-4dc4-ba80-c10e493a82fe" (UID: "8db0504f-55f6-4dc4-ba80-c10e493a82fe"). InnerVolumeSpecName "kube-api-access-qn6xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:33.802277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.802238    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn6xz\" (UniqueName: \"kubernetes.io/projected/8db0504f-55f6-4dc4-ba80-c10e493a82fe-kube-api-access-qn6xz\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.802277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:33.802268    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db0504f-55f6-4dc4-ba80-c10e493a82fe-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:34.185913 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.185877    2575 generic.go:356] "Generic (PLEG): container finished" podID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" containerID="65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92" exitCode=0
Jul 28 20:14:34.186110 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.185927    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"
Jul 28 20:14:34.186110 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.185963    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" event={"ID":"8db0504f-55f6-4dc4-ba80-c10e493a82fe","Type":"ContainerDied","Data":"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"}
Jul 28 20:14:34.186110 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.186014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v" event={"ID":"8db0504f-55f6-4dc4-ba80-c10e493a82fe","Type":"ContainerDied","Data":"8aaa432a115ab4ead532723593a5a113727f89c9dc0f2b882de0f8087c6e336f"}
Jul 28 20:14:34.186110 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.186040    2575 scope.go:117] "RemoveContainer" containerID="65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"
Jul 28 20:14:34.195867 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.195849    2575 scope.go:117] "RemoveContainer" containerID="65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"
Jul 28 20:14:34.196125 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:14:34.196106    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92\": container with ID starting with 65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92 not found: ID does not exist" containerID="65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"
Jul 28 20:14:34.196203 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.196135    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92"} err="failed to get container status \"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92\": rpc error: code = NotFound desc = could not find container \"65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92\": container with ID starting with 65e0387745b8b6abff3fed5f4344d22dfbf84a876ff1ce8d138c884235648d92 not found: ID does not exist"
Jul 28 20:14:34.207637 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.207614    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:14:34.211123 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.211097    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ba3d8d-d5cd4cb88-n8j6v"]
Jul 28 20:14:34.698495 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:34.698460    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" path="/var/lib/kubelet/pods/8db0504f-55f6-4dc4-ba80-c10e493a82fe/volumes"
Jul 28 20:14:40.178220 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:40.178177    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:14:40.178697 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:40.178418    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy" containerID="cri-o://e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8" gracePeriod=30
Jul 28 20:14:45.445828 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.445761    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:14:45.619035 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619003    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619052    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619077    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619111    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619159    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619276    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619312    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b59k\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619389    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:14:45.619473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy\") pod \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\" (UID: \"2a3c52b0-6e87-46f5-be04-dbd0583c27fb\") "
Jul 28 20:14:45.619899 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619641    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:14:45.619899 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619676    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:45.619899 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619835    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.619899 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619856    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.619899 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619871    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.620144 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.619940    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:14:45.620223 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.620201    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:14:45.622178 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.622149    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:45.622405 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.622373    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:14:45.622632 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.622614    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k" (OuterVolumeSpecName: "kube-api-access-2b59k") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "kube-api-access-2b59k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:45.623247 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.623222    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a3c52b0-6e87-46f5-be04-dbd0583c27fb" (UID: "2a3c52b0-6e87-46f5-be04-dbd0583c27fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721058    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721084    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721093    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b59k\" (UniqueName: \"kubernetes.io/projected/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-kube-api-access-2b59k\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721103    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721111    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:45.721128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:45.721119    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a3c52b0-6e87-46f5-be04-dbd0583c27fb-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:14:46.254134 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.254102    2575 generic.go:356] "Generic (PLEG): container finished" podID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerID="e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8" exitCode=0
Jul 28 20:14:46.254309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.254160    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"
Jul 28 20:14:46.254309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.254192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerDied","Data":"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"}
Jul 28 20:14:46.254309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.254268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4" event={"ID":"2a3c52b0-6e87-46f5-be04-dbd0583c27fb","Type":"ContainerDied","Data":"82a13b8c59dd08c4d3afa1dfe82cfa485c366822a13e9f47fcb51f75e76e2f4c"}
Jul 28 20:14:46.254309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.254291    2575 scope.go:117] "RemoveContainer" containerID="e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"
Jul 28 20:14:46.264231 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.264215    2575 scope.go:117] "RemoveContainer" containerID="13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"
Jul 28 20:14:46.271998 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.271979    2575 scope.go:117] "RemoveContainer" containerID="e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"
Jul 28 20:14:46.272236 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:14:46.272218    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8\": container with ID starting with e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8 not found: ID does not exist" containerID="e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"
Jul 28 20:14:46.272297 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.272243    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8"} err="failed to get container status \"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8\": rpc error: code = NotFound desc = could not find container \"e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8\": container with ID starting with e40e94e2e4857be874a7d2b5ec1cfb5890c33ed71e41124793b3f18c540689b8 not found: ID does not exist"
Jul 28 20:14:46.272297 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.272265    2575 scope.go:117] "RemoveContainer" containerID="13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"
Jul 28 20:14:46.272491 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:14:46.272474    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81\": container with ID starting with 13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81 not found: ID does not exist" containerID="13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"
Jul 28 20:14:46.272531 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.272496    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81"} err="failed to get container status \"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81\": rpc error: code = NotFound desc = could not find container \"13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81\": container with ID starting with 13c8f5534b9988b2466b617cf2b35f5260b3f41b5fea6cd75e309dfed2352d81 not found: ID does not exist"
Jul 28 20:14:46.278867 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.278844    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:14:46.284279 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.284253    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ba3d8d-openshift-default-658bdbdfb-c57l4"]
Jul 28 20:14:46.695847 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:14:46.695820    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" path="/var/lib/kubelet/pods/2a3c52b0-6e87-46f5-be04-dbd0583c27fb/volumes"
Jul 28 20:15:10.204215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204177    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204619    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204632    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204640    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" containerName="payload-pre-processing"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204646    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" containerName="payload-pre-processing"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204714    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204724    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:10.204753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.204747    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db0504f-55f6-4dc4-ba80-c10e493a82fe" containerName="payload-pre-processing"
Jul 28 20:15:10.207719 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.207700    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.210156 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.210131    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0559d0-openshift-default-dockercfg-5b2lj\""
Jul 28 20:15:10.223789 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.223761    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:15:10.245934 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.245906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246092 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.245938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246092 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.245963    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246177 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89kp8\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246177 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246254 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246333 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246268    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.246333 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.246324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347179 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89kp8\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347179 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347402 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347402 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347426    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347622 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347622 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347622 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347595    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347839 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347839 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.347948 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.347870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.348103 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.348074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.348252 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.348184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.349816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.349793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.350367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.350344    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.356182 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.356162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.356372 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.356352    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89kp8\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8\") pod \"e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") " pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.518652 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.518558    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:10.660703 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.660556    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:15:10.663687 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:15:10.663658    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565a6102_935f_46f7_bfa7_adbeb5fd0c72.slice/crio-2053486bd6d72adf2d15b1b00c8f8e753d5f4390a248c7468356e678e85cb0da WatchSource:0}: Error finding container 2053486bd6d72adf2d15b1b00c8f8e753d5f4390a248c7468356e678e85cb0da: Status 404 returned error can't find the container with id 2053486bd6d72adf2d15b1b00c8f8e753d5f4390a248c7468356e678e85cb0da
Jul 28 20:15:10.665804 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.665769    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:10.665911 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.665837    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:10.665911 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:10.665869    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:11.378067 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:11.378034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerStarted","Data":"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"}
Jul 28 20:15:11.378550 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:11.378091    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerStarted","Data":"2053486bd6d72adf2d15b1b00c8f8e753d5f4390a248c7468356e678e85cb0da"}
Jul 28 20:15:11.400349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:11.400299    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podStartSLOduration=1.400282453 podStartE2EDuration="1.400282453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:11.398918728 +0000 UTC m=+2395.287795710" watchObservedRunningTime="2026-07-28 20:15:11.400282453 +0000 UTC m=+2395.289159423"
Jul 28 20:15:11.519112 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:11.519073    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:11.524503 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:11.524475    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:12.383609 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:12.383573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:12.384627 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:12.384606    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:14.175992 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.175955    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:15:14.176406 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.176393    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:14.176447 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.176407    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3c52b0-6e87-46f5-be04-dbd0583c27fb" containerName="istio-proxy"
Jul 28 20:15:14.180042 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.180020    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.182517 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.182495    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0559d0\""
Jul 28 20:15:14.182630 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.182562    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0559d0-dockercfg-vng6z\""
Jul 28 20:15:14.192353 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.192317    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:15:14.288644 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.288601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v9zx\" (UniqueName: \"kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.289120 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.289096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.390309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.390277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v9zx\" (UniqueName: \"kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.390529 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.390378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.391289 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.391249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.398786 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.398757    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v9zx\" (UniqueName: \"kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx\") pod \"payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.493250 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.493166    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:14.640913 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:14.640881    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:15:14.642246 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:15:14.642219    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af950b3_6296_45b7_b655_265ca1fa07f6.slice/crio-af243d83f1f3aeba7ba6d239e4902416a789b540a064f622c114d10963967e49 WatchSource:0}: Error finding container af243d83f1f3aeba7ba6d239e4902416a789b540a064f622c114d10963967e49: Status 404 returned error can't find the container with id af243d83f1f3aeba7ba6d239e4902416a789b540a064f622c114d10963967e49
Jul 28 20:15:15.402786 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:15.402714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" event={"ID":"9af950b3-6296-45b7-b655-265ca1fa07f6","Type":"ContainerStarted","Data":"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"}
Jul 28 20:15:15.403156 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:15.402800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" event={"ID":"9af950b3-6296-45b7-b655-265ca1fa07f6","Type":"ContainerStarted","Data":"af243d83f1f3aeba7ba6d239e4902416a789b540a064f622c114d10963967e49"}
Jul 28 20:15:15.403156 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:15.402826    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:15.418572 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:15.418520    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" podStartSLOduration=1.418504324 podStartE2EDuration="1.418504324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:15.41702541 +0000 UTC m=+2399.305902384" watchObservedRunningTime="2026-07-28 20:15:15.418504324 +0000 UTC m=+2399.307381296"
Jul 28 20:15:16.716691 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:16.716661    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:15:16.719336 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:16.719310    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:15:26.410790 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:26.410755    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:15:28.424904 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:28.424855    2575 patch_prober.go:28] interesting pod/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 20:15:28.425281 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:28.424924    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:15:29.473913 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.473879    2575 generic.go:356] "Generic (PLEG): container finished" podID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerID="4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad" exitCode=0
Jul 28 20:15:29.474394 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.473939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerDied","Data":"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"}
Jul 28 20:15:29.474394 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.474349    2575 scope.go:117] "RemoveContainer" containerID="4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"
Jul 28 20:15:29.475566 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.475538    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:29.475640 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.475616    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:29.475683 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:29.475668    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:15:30.486926 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:30.485849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerStarted","Data":"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"}
Jul 28 20:15:30.519394 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:30.519361    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:30.519594 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:30.519571    2575 patch_prober.go:28] interesting pod/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 20:15:30.519655 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:30.519620    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 20:15:31.519947 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:31.519914    2575 patch_prober.go:28] interesting pod/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 20:15:31.520340 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:31.519964    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 20:15:32.519799 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:32.519767    2575 patch_prober.go:28] interesting pod/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 20:15:32.519987 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:32.519839    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 20:15:33.519419 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:33.519388    2575 patch_prober.go:28] interesting pod/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 20:15:33.519598 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:33.519438    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 20:15:34.522216 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:34.522179    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:34.522610 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:34.522571    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:34.523227 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:34.523211    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:15:41.522608 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.522523    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"]
Jul 28 20:15:41.525286 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.525269    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:41.527377 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.527356    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:15:41.533255 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.533225    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"]
Jul 28 20:15:41.565122 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.565087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f\" (UID: \"b23e8ebd-0837-4fc4-a889-1f685eb61651\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:41.666001 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.665961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f\" (UID: \"b23e8ebd-0837-4fc4-a889-1f685eb61651\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:41.666609 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.666582    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f\" (UID: \"b23e8ebd-0837-4fc4-a889-1f685eb61651\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:41.838317 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.838282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:41.970858 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:41.970835    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"]
Jul 28 20:15:41.972446 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:15:41.972418    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23e8ebd_0837_4fc4_a889_1f685eb61651.slice/crio-7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0 WatchSource:0}: Error finding container 7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0: Status 404 returned error can't find the container with id 7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0
Jul 28 20:15:42.556541 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:42.556446    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f" event={"ID":"b23e8ebd-0837-4fc4-a889-1f685eb61651","Type":"ContainerStarted","Data":"95cf9aaa5934e2a0304934212c8270a66020438687960ffc793e73ad26173ca0"}
Jul 28 20:15:42.556541 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:42.556512    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f" event={"ID":"b23e8ebd-0837-4fc4-a889-1f685eb61651","Type":"ContainerStarted","Data":"7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0"}
Jul 28 20:15:42.571998 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:42.571952    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f" podStartSLOduration=1.571936556 podStartE2EDuration="1.571936556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:42.570276339 +0000 UTC m=+2426.459153312" watchObservedRunningTime="2026-07-28 20:15:42.571936556 +0000 UTC m=+2426.460813526"
Jul 28 20:15:43.562400 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:43.562370    2575 generic.go:356] "Generic (PLEG): container finished" podID="b23e8ebd-0837-4fc4-a889-1f685eb61651" containerID="95cf9aaa5934e2a0304934212c8270a66020438687960ffc793e73ad26173ca0" exitCode=0
Jul 28 20:15:43.562805 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:43.562455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f" event={"ID":"b23e8ebd-0837-4fc4-a889-1f685eb61651","Type":"ContainerDied","Data":"95cf9aaa5934e2a0304934212c8270a66020438687960ffc793e73ad26173ca0"}
Jul 28 20:15:44.705378 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:44.705350    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:44.796944 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:44.796910    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca\") pod \"b23e8ebd-0837-4fc4-a889-1f685eb61651\" (UID: \"b23e8ebd-0837-4fc4-a889-1f685eb61651\") "
Jul 28 20:15:44.797774 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:44.797635    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b23e8ebd-0837-4fc4-a889-1f685eb61651" (UID: "b23e8ebd-0837-4fc4-a889-1f685eb61651"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:44.898860 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:44.898826    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b23e8ebd-0837-4fc4-a889-1f685eb61651-maas-api-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:15:45.573996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:45.573953    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f" event={"ID":"b23e8ebd-0837-4fc4-a889-1f685eb61651","Type":"ContainerDied","Data":"7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0"}
Jul 28 20:15:45.573996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:45.573989    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"
Jul 28 20:15:45.574209 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:45.574048    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fc38bc09c66afa6d82815bdaadc2ffb051c30e541e40801ae6cd28f5aaff3f0"
Jul 28 20:15:57.316043 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:57.316002    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"]
Jul 28 20:15:57.321960 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:57.321926    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6e8ba4-cgt2f"]
Jul 28 20:15:58.695678 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:15:58.695643    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23e8ebd-0837-4fc4-a889-1f685eb61651" path="/var/lib/kubelet/pods/b23e8ebd-0837-4fc4-a889-1f685eb61651/volumes"
Jul 28 20:16:13.664328 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:13.664292    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:16:13.665328 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:13.665266    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" podUID="9af950b3-6296-45b7-b655-265ca1fa07f6" containerName="payload-pre-processing" containerID="cri-o://c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b" gracePeriod=30
Jul 28 20:16:13.930914 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:13.930890    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:16:14.077780 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.077725    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v9zx\" (UniqueName: \"kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx\") pod \"9af950b3-6296-45b7-b655-265ca1fa07f6\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") "
Jul 28 20:16:14.077941 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.077804    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume\") pod \"9af950b3-6296-45b7-b655-265ca1fa07f6\" (UID: \"9af950b3-6296-45b7-b655-265ca1fa07f6\") "
Jul 28 20:16:14.078246 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.078221    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9af950b3-6296-45b7-b655-265ca1fa07f6" (UID: "9af950b3-6296-45b7-b655-265ca1fa07f6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:16:14.079848 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.079819    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx" (OuterVolumeSpecName: "kube-api-access-8v9zx") pod "9af950b3-6296-45b7-b655-265ca1fa07f6" (UID: "9af950b3-6296-45b7-b655-265ca1fa07f6"). InnerVolumeSpecName "kube-api-access-8v9zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:14.179688 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.179603    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9af950b3-6296-45b7-b655-265ca1fa07f6-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.179688 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.179633    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v9zx\" (UniqueName: \"kubernetes.io/projected/9af950b3-6296-45b7-b655-265ca1fa07f6-kube-api-access-8v9zx\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.729984 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.729943    2575 generic.go:356] "Generic (PLEG): container finished" podID="9af950b3-6296-45b7-b655-265ca1fa07f6" containerID="c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b" exitCode=0
Jul 28 20:16:14.730388 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.730005    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"
Jul 28 20:16:14.730388 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.730017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" event={"ID":"9af950b3-6296-45b7-b655-265ca1fa07f6","Type":"ContainerDied","Data":"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"}
Jul 28 20:16:14.730388 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.730096    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2" event={"ID":"9af950b3-6296-45b7-b655-265ca1fa07f6","Type":"ContainerDied","Data":"af243d83f1f3aeba7ba6d239e4902416a789b540a064f622c114d10963967e49"}
Jul 28 20:16:14.730388 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.730119    2575 scope.go:117] "RemoveContainer" containerID="c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"
Jul 28 20:16:14.738853 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.738828    2575 scope.go:117] "RemoveContainer" containerID="c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"
Jul 28 20:16:14.739090 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:16:14.739071    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b\": container with ID starting with c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b not found: ID does not exist" containerID="c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"
Jul 28 20:16:14.739158 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.739096    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b"} err="failed to get container status \"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b\": rpc error: code = NotFound desc = could not find container \"c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b\": container with ID starting with c7f775ead88e6b0ecf6849dc10a23064b8357c5f48bc12d62d785944ca8f8b8b not found: ID does not exist"
Jul 28 20:16:14.744827 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.744805    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:16:14.748642 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:14.748622    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0559d0-bd9587884-mqtr2"]
Jul 28 20:16:16.695062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:16.695033    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9af950b3-6296-45b7-b655-265ca1fa07f6" path="/var/lib/kubelet/pods/9af950b3-6296-45b7-b655-265ca1fa07f6/volumes"
Jul 28 20:16:20.206528 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:20.206493    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:16:20.206988 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:20.206716    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy" containerID="cri-o://e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c" gracePeriod=30
Jul 28 20:16:25.472224 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.472157    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:16:25.577130 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577100    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577315 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577142    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577315 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577169    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577315 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577199    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577315 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577540 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577325    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577540 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577346    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577540 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577375    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89kp8\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577540 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577409    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert\") pod \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\" (UID: \"565a6102-935f-46f7-bfa7-adbeb5fd0c72\") "
Jul 28 20:16:25.577540 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:25.577840 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.577781    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:25.578130 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:16:25.578245 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578207    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.578245 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578229    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.578245 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578243    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.578421 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578256    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:25.578479 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.578446    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data" (OuterVolumeSpecName: "istio-data") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:25.579909 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.579866    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:25.579909 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.579894    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8" (OuterVolumeSpecName: "kube-api-access-89kp8") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "kube-api-access-89kp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:25.580040 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.579933    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:16:25.580040 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.579938    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token" (OuterVolumeSpecName: "istio-token") pod "565a6102-935f-46f7-bfa7-adbeb5fd0c72" (UID: "565a6102-935f-46f7-bfa7-adbeb5fd0c72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:25.679192 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679162    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.679192 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679188    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.679349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679198    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89kp8\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-kube-api-access-89kp8\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.679349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679209    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.679349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679218    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.679349 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.679225    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/565a6102-935f-46f7-bfa7-adbeb5fd0c72-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:16:25.793682 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.793590    2575 generic.go:356] "Generic (PLEG): container finished" podID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerID="e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c" exitCode=0
Jul 28 20:16:25.793682 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.793643    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"
Jul 28 20:16:25.793931 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.793636    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerDied","Data":"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"}
Jul 28 20:16:25.793931 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.793792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq" event={"ID":"565a6102-935f-46f7-bfa7-adbeb5fd0c72","Type":"ContainerDied","Data":"2053486bd6d72adf2d15b1b00c8f8e753d5f4390a248c7468356e678e85cb0da"}
Jul 28 20:16:25.793931 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.793827    2575 scope.go:117] "RemoveContainer" containerID="e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"
Jul 28 20:16:25.803027 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.803008    2575 scope.go:117] "RemoveContainer" containerID="4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"
Jul 28 20:16:25.812791 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.812772    2575 scope.go:117] "RemoveContainer" containerID="e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"
Jul 28 20:16:25.813054 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:16:25.813033    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c\": container with ID starting with e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c not found: ID does not exist" containerID="e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"
Jul 28 20:16:25.813106 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.813062    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c"} err="failed to get container status \"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c\": rpc error: code = NotFound desc = could not find container \"e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c\": container with ID starting with e4f192e260f9b001c4a1f8c92e48248791defd4bdcc2cb84e8eec34a5743437c not found: ID does not exist"
Jul 28 20:16:25.813106 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.813079    2575 scope.go:117] "RemoveContainer" containerID="4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"
Jul 28 20:16:25.813302 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:16:25.813284    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad\": container with ID starting with 4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad not found: ID does not exist" containerID="4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"
Jul 28 20:16:25.813354 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.813314    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad"} err="failed to get container status \"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad\": rpc error: code = NotFound desc = could not find container \"4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad\": container with ID starting with 4dfe1f037654c74869942ce90023b23bfa99f33cd3507e6829653756134b69ad not found: ID does not exist"
Jul 28 20:16:25.816288 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.816267    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:16:25.822021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:25.822001    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0559d0-openshift-default-8675fcd6b8-sg8nq"]
Jul 28 20:16:26.695538 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:26.695502    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" path="/var/lib/kubelet/pods/565a6102-935f-46f7-bfa7-adbeb5fd0c72/volumes"
Jul 28 20:16:49.454055 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454021    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454438    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454452    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454466    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af950b3-6296-45b7-b655-265ca1fa07f6" containerName="payload-pre-processing"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454471    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af950b3-6296-45b7-b655-265ca1fa07f6" containerName="payload-pre-processing"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454480    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23e8ebd-0837-4fc4-a889-1f685eb61651" containerName="revoke-keys"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454486    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23e8ebd-0837-4fc4-a889-1f685eb61651" containerName="revoke-keys"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454503    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454511    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454566    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af950b3-6296-45b7-b655-265ca1fa07f6" containerName="payload-pre-processing"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454575    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:49.454625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.454587    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23e8ebd-0837-4fc4-a889-1f685eb61651" containerName="revoke-keys"
Jul 28 20:16:49.459397 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.459373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.461671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.461646    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-60efb4-openshift-default-dockercfg-8l8wk\""
Jul 28 20:16:49.470277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.470251    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:16:49.601617 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601582    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkfjd\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601648    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601816 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.601996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.601927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702432 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702432 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702671 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702894 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkfjd\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.702894 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.702707    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.703084 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.703058    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.703171 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.703093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.703222 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.703191    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.703273 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.703252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.703330 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.703314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.705122 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.705073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.705266 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.705248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.711643 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.711612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkfjd\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.712491 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.712468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token\") pod \"e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") " pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.770601 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.770544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:49.912809 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.912785    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:16:49.914518 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:16:49.914489    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b6aad6_8ebe_4fc3_aea9_883214823c5c.slice/crio-51699ade76c4a6f2d2a951f59f35edf860c191643604215f4b9c14c035f53972 WatchSource:0}: Error finding container 51699ade76c4a6f2d2a951f59f35edf860c191643604215f4b9c14c035f53972: Status 404 returned error can't find the container with id 51699ade76c4a6f2d2a951f59f35edf860c191643604215f4b9c14c035f53972
Jul 28 20:16:49.916929 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.916894    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:16:49.917034 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.916973    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:16:49.917034 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:49.917021    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:16:50.926744 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:50.926697    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerStarted","Data":"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"}
Jul 28 20:16:50.927121 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:50.926763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerStarted","Data":"51699ade76c4a6f2d2a951f59f35edf860c191643604215f4b9c14c035f53972"}
Jul 28 20:16:50.949609 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:50.949559    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podStartSLOduration=1.9495452210000002 podStartE2EDuration="1.949545221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:50.947157183 +0000 UTC m=+2494.836034157" watchObservedRunningTime="2026-07-28 20:16:50.949545221 +0000 UTC m=+2494.838422192"
Jul 28 20:16:51.771765 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:51.771711    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:51.776839 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:51.776812    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:51.932137 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:51.932097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:51.932938 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:51.932918    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:16:53.984915 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:53.984881    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:16:53.985368 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:53.985354    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="565a6102-935f-46f7-bfa7-adbeb5fd0c72" containerName="istio-proxy"
Jul 28 20:16:53.988307 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:53.988289    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:53.992087 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:53.992062    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-60efb4-dockercfg-88m8n\""
Jul 28 20:16:53.992221 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:53.992167    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-60efb4\""
Jul 28 20:16:54.004818 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.004796    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:16:54.148139 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.148103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhzl\" (UniqueName: \"kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.148139 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.148141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.249135 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.249039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhzl\" (UniqueName: \"kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.249135 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.249080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.249841 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.249811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.266261 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.266220    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhzl\" (UniqueName: \"kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl\") pod \"payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.298717 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.298676    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.464007 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.463981    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:16:54.466356 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:16:54.466327    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178fd2e1_279f_4936_abee_0452a19bfdda.slice/crio-48574ff59a44caea752f3a1b9a03413e9e58a2ce4a84752829e343f05b32f7c9 WatchSource:0}: Error finding container 48574ff59a44caea752f3a1b9a03413e9e58a2ce4a84752829e343f05b32f7c9: Status 404 returned error can't find the container with id 48574ff59a44caea752f3a1b9a03413e9e58a2ce4a84752829e343f05b32f7c9
Jul 28 20:16:54.952142 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.952107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" event={"ID":"178fd2e1-279f-4936-abee-0452a19bfdda","Type":"ContainerStarted","Data":"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"}
Jul 28 20:16:54.952317 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.952171    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:16:54.952317 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.952196    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" event={"ID":"178fd2e1-279f-4936-abee-0452a19bfdda","Type":"ContainerStarted","Data":"48574ff59a44caea752f3a1b9a03413e9e58a2ce4a84752829e343f05b32f7c9"}
Jul 28 20:16:54.974556 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:16:54.974497    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" podStartSLOduration=1.974478529 podStartE2EDuration="1.974478529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:54.97186235 +0000 UTC m=+2498.860739323" watchObservedRunningTime="2026-07-28 20:16:54.974478529 +0000 UTC m=+2498.863355501"
Jul 28 20:17:05.960146 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:05.960113    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:17:07.973929 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:07.973878    2575 patch_prober.go:28] interesting pod/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 20:17:07.974425 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:07.974072    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:17:09.034260 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.034227    2575 generic.go:356] "Generic (PLEG): container finished" podID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerID="f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865" exitCode=0
Jul 28 20:17:09.034698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.034283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerDied","Data":"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"}
Jul 28 20:17:09.034769 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.034696    2575 scope.go:117] "RemoveContainer" containerID="f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"
Jul 28 20:17:09.035742 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.035694    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:17:09.035821 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.035784    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:17:09.035863 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:09.035819    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:17:10.042660 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:10.042622    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerStarted","Data":"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"}
Jul 28 20:17:10.771466 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:10.771424    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:17:10.771775 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:10.771746    2575 patch_prober.go:28] interesting pod/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 28 20:17:10.771884 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:10.771800    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 28 20:17:11.772037 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:11.771991    2575 patch_prober.go:28] interesting pod/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 28 20:17:11.772542 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:11.772062    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 28 20:17:12.771837 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:12.771796    2575 patch_prober.go:28] interesting pod/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 28 20:17:12.772114 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:12.772086    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 28 20:17:13.771190 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:13.771150    2575 patch_prober.go:28] interesting pod/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 28 20:17:13.771484 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:13.771454    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 28 20:17:14.775624 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:14.775587    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:17:14.776082 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:14.775961    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:17:14.776753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:17:14.776715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:20:16.748639 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:20:16.748563    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:20:16.752785 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:20:16.752766    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:22:17.598186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:22:17.598155    2575 scope.go:117] "RemoveContainer" containerID="95cf9aaa5934e2a0304934212c8270a66020438687960ffc793e73ad26173ca0"
Jul 28 20:25:16.780720 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:25:16.780690    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:25:16.786495 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:25:16.786470    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:26:41.169098 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.169062    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:26:41.173560 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.173535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.176598 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.176572    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d7a9b9-serving-cert\""
Jul 28 20:26:41.189134 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.189102    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:26:41.277085 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.277024    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbkmp\" (UniqueName: \"kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.277266 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.277133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.277266 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.277198    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.378485 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.378443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.378698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.378511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.378698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.378599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbkmp\" (UniqueName: \"kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.378698 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:26:41.378661    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-serving-cert: secret "maas-api-e2e-ipp-a-d7a9b9-serving-cert" not found
Jul 28 20:26:41.378909 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:26:41.378753    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls podName:ac1779bf-08cc-4a92-bf41-7341141416c9 nodeName:}" failed. No retries permitted until 2026-07-28 20:26:41.878720249 +0000 UTC m=+3085.767597202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls") pod "maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" (UID: "ac1779bf-08cc-4a92-bf41-7341141416c9") : secret "maas-api-e2e-ipp-a-d7a9b9-serving-cert" not found
Jul 28 20:26:41.379163 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.379144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.393228 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.393199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbkmp\" (UniqueName: \"kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.884165 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.884129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:41.886653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:41.886628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:42.085232 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:42.085199    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:42.227612 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:42.227583    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:26:42.229032 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:26:42.229006    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1779bf_08cc_4a92_bf41_7341141416c9.slice/crio-ebd0ae9011b0fc62ac88e00f64406deec321cc1fdfde4de368570f46c6d4207d WatchSource:0}: Error finding container ebd0ae9011b0fc62ac88e00f64406deec321cc1fdfde4de368570f46c6d4207d: Status 404 returned error can't find the container with id ebd0ae9011b0fc62ac88e00f64406deec321cc1fdfde4de368570f46c6d4207d
Jul 28 20:26:42.230446 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:42.230428    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:26:42.983929 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:42.983896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" event={"ID":"ac1779bf-08cc-4a92-bf41-7341141416c9","Type":"ContainerStarted","Data":"ebd0ae9011b0fc62ac88e00f64406deec321cc1fdfde4de368570f46c6d4207d"}
Jul 28 20:26:43.991045 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:43.991009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" event={"ID":"ac1779bf-08cc-4a92-bf41-7341141416c9","Type":"ContainerStarted","Data":"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"}
Jul 28 20:26:43.991516 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:43.991149    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:26:44.011352 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:44.010866    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" podStartSLOduration=2.081048838 podStartE2EDuration="3.010850009s" totalImagesPullingTime="929.801171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:44.008496949 +0000 UTC m=+3087.897373937" watchObservedRunningTime="2026-07-28 20:26:44.010850009 +0000 UTC m=+3087.899726980"
Jul 28 20:26:50.001238 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:26:50.001210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:27:11.115348 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.115312    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:27:11.118564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.118547    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.121031 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.120995    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f11e93\""
Jul 28 20:27:11.121167 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.121069    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f11e93-dockercfg-99c4p\""
Jul 28 20:27:11.127235 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.127208    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:27:11.274073 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.274034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b727s\" (UniqueName: \"kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.274253 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.274107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.375023 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.374921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.375201 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.375027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b727s\" (UniqueName: \"kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.375605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.375583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.383849 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.383815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b727s\" (UniqueName: \"kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s\") pod \"payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.429868 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.429832    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:11.802989 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:27:11.802949    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5121c9_6490_439f_8085_a759953c399f.slice/crio-a97bb8a1a410ad9ca51dfd8b8d8d131880ad55385fabe3303d70b3a36cb04930 WatchSource:0}: Error finding container a97bb8a1a410ad9ca51dfd8b8d8d131880ad55385fabe3303d70b3a36cb04930: Status 404 returned error can't find the container with id a97bb8a1a410ad9ca51dfd8b8d8d131880ad55385fabe3303d70b3a36cb04930
Jul 28 20:27:11.806166 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:11.806133    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:27:12.144702 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:12.144664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" event={"ID":"ec5121c9-6490-439f-8085-a759953c399f","Type":"ContainerStarted","Data":"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"}
Jul 28 20:27:12.145207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:12.144724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" event={"ID":"ec5121c9-6490-439f-8085-a759953c399f","Type":"ContainerStarted","Data":"a97bb8a1a410ad9ca51dfd8b8d8d131880ad55385fabe3303d70b3a36cb04930"}
Jul 28 20:27:12.145207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:12.144800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:12.162063 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:12.162013    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" podStartSLOduration=1.16199713 podStartE2EDuration="1.16199713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:12.161166265 +0000 UTC m=+3116.050043237" watchObservedRunningTime="2026-07-28 20:27:12.16199713 +0000 UTC m=+3116.050874101"
Jul 28 20:27:23.152202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:23.152168    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:27:42.240248 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:27:42.240217    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"
Jul 28 20:28:48.001596 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:48.001566    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"
Jul 28 20:28:48.862564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:48.862528    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:28:48.866373 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:48.866347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:48.875941 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:48.875905    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-635813-openshift-default-dockercfg-46ncg\""
Jul 28 20:28:48.899979 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:48.899947    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:28:49.022192 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdqc\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.022599 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.022453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123185 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123092    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdqc\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123185 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123185 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123201    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123400    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123494 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123814 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123578    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123814 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123915 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123847    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.123915 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.124017 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.123997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.124057 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.124036    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.125508 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.125487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.126016 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.125994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.133258 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.133235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.133345 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.133239    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdqc\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc\") pod \"e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") " pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.183211 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.183185    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:49.343984 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.343947    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:28:49.346538 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:28:49.346512    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74390e16_83ca_4823_86f9_ee4f0f9e5a79.slice/crio-e3df3ca5f28005793287cc0eb56a5159a9234a4bc7bc98e4a228869ae4cff5b9 WatchSource:0}: Error finding container e3df3ca5f28005793287cc0eb56a5159a9234a4bc7bc98e4a228869ae4cff5b9: Status 404 returned error can't find the container with id e3df3ca5f28005793287cc0eb56a5159a9234a4bc7bc98e4a228869ae4cff5b9
Jul 28 20:28:49.348798 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.348762    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:28:49.348882 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.348829    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:28:49.348882 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.348858    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:28:49.647600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.647505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerStarted","Data":"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"}
Jul 28 20:28:49.647600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.647560    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerStarted","Data":"e3df3ca5f28005793287cc0eb56a5159a9234a4bc7bc98e4a228869ae4cff5b9"}
Jul 28 20:28:49.679455 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:49.679401    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podStartSLOduration=1.6793860170000001 podStartE2EDuration="1.679386017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:49.676069389 +0000 UTC m=+3213.564946360" watchObservedRunningTime="2026-07-28 20:28:49.679386017 +0000 UTC m=+3213.568262988"
Jul 28 20:28:50.184086 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:50.184047    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:50.189444 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:50.189420    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:50.652957 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:50.652926    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:50.654009 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:50.653987    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:28:51.874600 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.874564    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:28:51.878387 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.878367    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:51.881484 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.881463    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-635813-serving-cert\""
Jul 28 20:28:51.893797 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.893774    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:28:51.955213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.955177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:51.955376 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.955302    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:51.955376 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:51.955365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57xp\" (UniqueName: \"kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.056831 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.056800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.056977 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.056857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.056977 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.056887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57xp\" (UniqueName: \"kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.057071 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:28:52.057003    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-serving-cert: secret "maas-api-e2e-ipp-cleanup-635813-serving-cert" not found
Jul 28 20:28:52.057071 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:28:52.057065    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls podName:180f026b-cd4a-47f6-855a-f9a78ff87848 nodeName:}" failed. No retries permitted until 2026-07-28 20:28:52.557049169 +0000 UTC m=+3216.445926117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" (UID: "180f026b-cd4a-47f6-855a-f9a78ff87848") : secret "maas-api-e2e-ipp-cleanup-635813-serving-cert" not found
Jul 28 20:28:52.057399 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.057381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.086866 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.086838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57xp\" (UniqueName: \"kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.561906 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.561874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.564279 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.564252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.789908 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.789877    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:52.970527 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:52.970488    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:28:52.971340 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:28:52.971292    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180f026b_cd4a_47f6_855a_f9a78ff87848.slice/crio-f80017eec7ac90cba7dcf045b6e5a377b6dd040c6c0f7b9ae9a127c8562dacf1 WatchSource:0}: Error finding container f80017eec7ac90cba7dcf045b6e5a377b6dd040c6c0f7b9ae9a127c8562dacf1: Status 404 returned error can't find the container with id f80017eec7ac90cba7dcf045b6e5a377b6dd040c6c0f7b9ae9a127c8562dacf1
Jul 28 20:28:53.396624 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.396589    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:28:53.401432 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.401412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.409064 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.409044    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-635813\""
Jul 28 20:28:53.409311 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.409295    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-635813-dockercfg-h6mzr\""
Jul 28 20:28:53.436081 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.436056    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:28:53.572326 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.572291    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.572449 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.572435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9kf\" (UniqueName: \"kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.669133 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.669051    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" event={"ID":"180f026b-cd4a-47f6-855a-f9a78ff87848","Type":"ContainerStarted","Data":"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"}
Jul 28 20:28:53.669133 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.669108    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" event={"ID":"180f026b-cd4a-47f6-855a-f9a78ff87848","Type":"ContainerStarted","Data":"f80017eec7ac90cba7dcf045b6e5a377b6dd040c6c0f7b9ae9a127c8562dacf1"}
Jul 28 20:28:53.669324 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.669208    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:28:53.673791 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.673766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.673933 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.673915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9kf\" (UniqueName: \"kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.674451 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.674433    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.690906 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.690880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9kf\" (UniqueName: \"kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf\") pod \"payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:53.708945 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.708895    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" podStartSLOduration=2.157530373 podStartE2EDuration="2.708879919s" totalImagesPullingTime="551.349546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:53.708312646 +0000 UTC m=+3217.597189618" watchObservedRunningTime="2026-07-28 20:28:53.708879919 +0000 UTC m=+3217.597756890"
Jul 28 20:28:53.711415 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:53.711385    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:54.095404 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:54.095378    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:28:54.097843 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:28:54.097807    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1b86ed_ac6f_47c9_a77e_3ab240e0f8a2.slice/crio-9d07cd1e23dfde4e12626ff5218a5e62f7dcb4f93172dbf968e9fb7d5ff265bf WatchSource:0}: Error finding container 9d07cd1e23dfde4e12626ff5218a5e62f7dcb4f93172dbf968e9fb7d5ff265bf: Status 404 returned error can't find the container with id 9d07cd1e23dfde4e12626ff5218a5e62f7dcb4f93172dbf968e9fb7d5ff265bf
Jul 28 20:28:54.678001 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:54.677955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" event={"ID":"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2","Type":"ContainerStarted","Data":"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"}
Jul 28 20:28:54.678215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:54.678026    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" event={"ID":"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2","Type":"ContainerStarted","Data":"9d07cd1e23dfde4e12626ff5218a5e62f7dcb4f93172dbf968e9fb7d5ff265bf"}
Jul 28 20:28:54.678215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:54.678056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:28:54.697228 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:54.697180    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" podStartSLOduration=1.697165899 podStartE2EDuration="1.697165899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:54.696747917 +0000 UTC m=+3218.585624881" watchObservedRunningTime="2026-07-28 20:28:54.697165899 +0000 UTC m=+3218.586042869"
Jul 28 20:28:59.691103 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:28:59.691056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:29:05.686715 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:05.686686    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:29:06.354190 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.354022    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": read tcp 10.134.0.2:46128->10.134.0.60:15021: read: connection reset by peer" start-of-body=
Jul 28 20:29:06.354190 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.354088    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": read tcp 10.134.0.2:46128->10.134.0.60:15021: read: connection reset by peer"
Jul 28 20:29:06.744098 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.744014    2575 generic.go:356] "Generic (PLEG): container finished" podID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerID="506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a" exitCode=0
Jul 28 20:29:06.744098 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.744084    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerDied","Data":"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"}
Jul 28 20:29:06.744515 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.744490    2575 scope.go:117] "RemoveContainer" containerID="506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"
Jul 28 20:29:06.745576 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.745547    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:29:06.745673 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.745612    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:29:06.745673 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:06.745643    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 20:29:07.751341 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:07.751307    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerStarted","Data":"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"}
Jul 28 20:29:08.184098 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:08.184060    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:08.184492 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:08.184404    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 28 20:29:08.184706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:08.184683    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 28 20:29:09.184367 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:09.184321    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 28 20:29:09.184935 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:09.184399    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 28 20:29:10.184037 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:10.183997    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 28 20:29:10.184414 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:10.184381    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 28 20:29:11.184513 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:11.184471    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 28 20:29:11.185066 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:11.184546    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 28 20:29:12.187544 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:12.187511    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:12.188051 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:12.187949    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:12.188589 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:12.188571    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:22.819463 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:22.819420    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:29:22.819956 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:22.819689    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" podUID="180f026b-cd4a-47f6-855a-f9a78ff87848" containerName="maas-api" containerID="cri-o://b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582" gracePeriod=30
Jul 28 20:29:22.941527 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:22.941497    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:29:22.941947 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:22.941914    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" podUID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" containerName="payload-pre-processing" containerID="cri-o://78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863" gracePeriod=30
Jul 28 20:29:23.094230 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.094187    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:29:23.171795 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.170904    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z57xp\" (UniqueName: \"kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp\") pod \"180f026b-cd4a-47f6-855a-f9a78ff87848\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") "
Jul 28 20:29:23.171795 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.171082    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca\") pod \"180f026b-cd4a-47f6-855a-f9a78ff87848\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") "
Jul 28 20:29:23.171795 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.171129    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") pod \"180f026b-cd4a-47f6-855a-f9a78ff87848\" (UID: \"180f026b-cd4a-47f6-855a-f9a78ff87848\") "
Jul 28 20:29:23.178087 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.177669    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp" (OuterVolumeSpecName: "kube-api-access-z57xp") pod "180f026b-cd4a-47f6-855a-f9a78ff87848" (UID: "180f026b-cd4a-47f6-855a-f9a78ff87848"). InnerVolumeSpecName "kube-api-access-z57xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:23.178087 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.178041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "180f026b-cd4a-47f6-855a-f9a78ff87848" (UID: "180f026b-cd4a-47f6-855a-f9a78ff87848"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:23.180930 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.180866    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "180f026b-cd4a-47f6-855a-f9a78ff87848" (UID: "180f026b-cd4a-47f6-855a-f9a78ff87848"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:29:23.216204 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.216179    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:29:23.272050 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.272022    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/180f026b-cd4a-47f6-855a-f9a78ff87848-openshift-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.272050 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.272046    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/180f026b-cd4a-47f6-855a-f9a78ff87848-maas-api-tls\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.272050 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.272056    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z57xp\" (UniqueName: \"kubernetes.io/projected/180f026b-cd4a-47f6-855a-f9a78ff87848-kube-api-access-z57xp\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.373340 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.373251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume\") pod \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") "
Jul 28 20:29:23.373340 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.373335    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd9kf\" (UniqueName: \"kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf\") pod \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\" (UID: \"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2\") "
Jul 28 20:29:23.373689 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.373664    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" (UID: "6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:23.375323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.375304    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf" (OuterVolumeSpecName: "kube-api-access-pd9kf") pod "6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" (UID: "6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2"). InnerVolumeSpecName "kube-api-access-pd9kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:23.474472 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.474437    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.474472 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.474465    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd9kf\" (UniqueName: \"kubernetes.io/projected/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2-kube-api-access-pd9kf\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.848318 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.848288    2575 generic.go:356] "Generic (PLEG): container finished" podID="180f026b-cd4a-47f6-855a-f9a78ff87848" containerID="b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582" exitCode=0
Jul 28 20:29:23.848706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.848344    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"
Jul 28 20:29:23.848706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.848373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" event={"ID":"180f026b-cd4a-47f6-855a-f9a78ff87848","Type":"ContainerDied","Data":"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"}
Jul 28 20:29:23.848706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.848450    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z" event={"ID":"180f026b-cd4a-47f6-855a-f9a78ff87848","Type":"ContainerDied","Data":"f80017eec7ac90cba7dcf045b6e5a377b6dd040c6c0f7b9ae9a127c8562dacf1"}
Jul 28 20:29:23.848706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.848467    2575 scope.go:117] "RemoveContainer" containerID="b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"
Jul 28 20:29:23.849868 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.849844    2575 generic.go:356] "Generic (PLEG): container finished" podID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" containerID="78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863" exitCode=0
Jul 28 20:29:23.849980 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.849883    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" event={"ID":"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2","Type":"ContainerDied","Data":"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"}
Jul 28 20:29:23.849980 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.849905    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"
Jul 28 20:29:23.849980 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.849924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth" event={"ID":"6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2","Type":"ContainerDied","Data":"9d07cd1e23dfde4e12626ff5218a5e62f7dcb4f93172dbf968e9fb7d5ff265bf"}
Jul 28 20:29:23.857987 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.857873    2575 scope.go:117] "RemoveContainer" containerID="b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"
Jul 28 20:29:23.858253 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:29:23.858228    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582\": container with ID starting with b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582 not found: ID does not exist" containerID="b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"
Jul 28 20:29:23.858330 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.858262    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582"} err="failed to get container status \"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582\": rpc error: code = NotFound desc = could not find container \"b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582\": container with ID starting with b61b43005ee50e81bc78a3f551482cbfd8541b1d5ef9fba6791db3641155d582 not found: ID does not exist"
Jul 28 20:29:23.858330 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.858284    2575 scope.go:117] "RemoveContainer" containerID="78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"
Jul 28 20:29:23.866420 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.866400    2575 scope.go:117] "RemoveContainer" containerID="78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"
Jul 28 20:29:23.866673 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:29:23.866653    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863\": container with ID starting with 78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863 not found: ID does not exist" containerID="78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"
Jul 28 20:29:23.866737 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.866687    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863"} err="failed to get container status \"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863\": rpc error: code = NotFound desc = could not find container \"78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863\": container with ID starting with 78da248f18cdb42d552a40ada28ba9208fa015f16a5ab0624ff93f6804fb0863 not found: ID does not exist"
Jul 28 20:29:23.877338 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.877304    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:29:23.880506 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.880481    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-635813-5c64994fd8-nrsth"]
Jul 28 20:29:23.895339 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.895313    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:29:23.899524 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:23.899503    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-635813-67b856797-j8f9z"]
Jul 28 20:29:24.696633 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:24.696603    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="180f026b-cd4a-47f6-855a-f9a78ff87848" path="/var/lib/kubelet/pods/180f026b-cd4a-47f6-855a-f9a78ff87848/volumes"
Jul 28 20:29:24.697006 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:24.696990    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" path="/var/lib/kubelet/pods/6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2/volumes"
Jul 28 20:29:29.801309 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:29.801273    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:29:29.801790 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:29.801471    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy" containerID="cri-o://15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9" gracePeriod=30
Jul 28 20:29:33.793420 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.793380    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"]
Jul 28 20:29:33.793933 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.793922    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="180f026b-cd4a-47f6-855a-f9a78ff87848" containerName="maas-api"
Jul 28 20:29:33.793979 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.793936    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="180f026b-cd4a-47f6-855a-f9a78ff87848" containerName="maas-api"
Jul 28 20:29:33.793979 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.793957    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" containerName="payload-pre-processing"
Jul 28 20:29:33.794054 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.793984    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" containerName="payload-pre-processing"
Jul 28 20:29:33.794091 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.794071    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1b86ed-ac6f-47c9-a77e-3ab240e0f8a2" containerName="payload-pre-processing"
Jul 28 20:29:33.794091 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.794081    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="180f026b-cd4a-47f6-855a-f9a78ff87848" containerName="maas-api"
Jul 28 20:29:33.797059 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.797043    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:33.806473 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.806453    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:29:33.833464 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.833436    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"]
Jul 28 20:29:33.976805 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:33.976775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n\" (UID: \"40e50461-4adb-410a-a829-d0b5c645897f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:34.077779 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.077742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n\" (UID: \"40e50461-4adb-410a-a829-d0b5c645897f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:34.078285 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.078266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n\" (UID: \"40e50461-4adb-410a-a829-d0b5c645897f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:34.107212 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.107185    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:34.270815 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.270786    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"]
Jul 28 20:29:34.273934 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:29:34.273903    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e50461_4adb_410a_a829_d0b5c645897f.slice/crio-dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c WatchSource:0}: Error finding container dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c: Status 404 returned error can't find the container with id dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c
Jul 28 20:29:34.916871 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.916829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n" event={"ID":"40e50461-4adb-410a-a829-d0b5c645897f","Type":"ContainerStarted","Data":"2d4625e43264683de2a108cec1e8ee505738d2c5cca8597a18685c5b8d1da6d4"}
Jul 28 20:29:34.917221 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.916885    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n" event={"ID":"40e50461-4adb-410a-a829-d0b5c645897f","Type":"ContainerStarted","Data":"dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c"}
Jul 28 20:29:34.936149 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:34.936101    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n" podStartSLOduration=1.9360858749999998 podStartE2EDuration="1.936085875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:34.933948532 +0000 UTC m=+3258.822825505" watchObservedRunningTime="2026-07-28 20:29:34.936085875 +0000 UTC m=+3258.824962846"
Jul 28 20:29:35.045352 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.045322    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:35.188905 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.188876    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.188998 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.188917    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.188998 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.188949    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189075 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189006    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189075 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189046    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189075 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sdqc\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189083    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189202 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189151    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189303 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189200    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy\") pod \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\" (UID: \"74390e16-83ca-4823-86f9-ee4f0f9e5a79\") "
Jul 28 20:29:35.189303 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:35.189440 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189415    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:35.189607 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189587    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.189696 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189613    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.189696 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:35.189844 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189806    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:35.189913 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.189856    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data" (OuterVolumeSpecName: "istio-data") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:35.191138 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.191114    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:29:35.191332 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.191310    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc" (OuterVolumeSpecName: "kube-api-access-2sdqc") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "kube-api-access-2sdqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:35.191685 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.191668    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:35.191747 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.191681    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token" (OuterVolumeSpecName: "istio-token") pod "74390e16-83ca-4823-86f9-ee4f0f9e5a79" (UID: "74390e16-83ca-4823-86f9-ee4f0f9e5a79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:35.291181 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291143    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sdqc\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-kube-api-access-2sdqc\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291181 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291174    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291181 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291185    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291401 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291195    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291401 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291207    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291401 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291215    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.291401 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.291222    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74390e16-83ca-4823-86f9-ee4f0f9e5a79-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:35.924142 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.924101    2575 generic.go:356] "Generic (PLEG): container finished" podID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerID="15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9" exitCode=0
Jul 28 20:29:35.924653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.924153    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"
Jul 28 20:29:35.924653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.924193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerDied","Data":"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"}
Jul 28 20:29:35.924653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.924272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw" event={"ID":"74390e16-83ca-4823-86f9-ee4f0f9e5a79","Type":"ContainerDied","Data":"e3df3ca5f28005793287cc0eb56a5159a9234a4bc7bc98e4a228869ae4cff5b9"}
Jul 28 20:29:35.924653 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.924308    2575 scope.go:117] "RemoveContainer" containerID="15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"
Jul 28 20:29:35.926110 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.926085    2575 generic.go:356] "Generic (PLEG): container finished" podID="40e50461-4adb-410a-a829-d0b5c645897f" containerID="2d4625e43264683de2a108cec1e8ee505738d2c5cca8597a18685c5b8d1da6d4" exitCode=0
Jul 28 20:29:35.926226 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.926165    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n" event={"ID":"40e50461-4adb-410a-a829-d0b5c645897f","Type":"ContainerDied","Data":"2d4625e43264683de2a108cec1e8ee505738d2c5cca8597a18685c5b8d1da6d4"}
Jul 28 20:29:35.934173 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.934154    2575 scope.go:117] "RemoveContainer" containerID="506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"
Jul 28 20:29:35.942168 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.942149    2575 scope.go:117] "RemoveContainer" containerID="15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"
Jul 28 20:29:35.942424 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:29:35.942405    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9\": container with ID starting with 15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9 not found: ID does not exist" containerID="15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"
Jul 28 20:29:35.942495 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.942436    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9"} err="failed to get container status \"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9\": rpc error: code = NotFound desc = could not find container \"15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9\": container with ID starting with 15c07980a1541f76548f56b3a35dec50379d739f4dc18209620d5c0f15bd15e9 not found: ID does not exist"
Jul 28 20:29:35.942495 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.942460    2575 scope.go:117] "RemoveContainer" containerID="506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"
Jul 28 20:29:35.942701 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:29:35.942676    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a\": container with ID starting with 506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a not found: ID does not exist" containerID="506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"
Jul 28 20:29:35.942759 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.942706    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a"} err="failed to get container status \"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a\": rpc error: code = NotFound desc = could not find container \"506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a\": container with ID starting with 506b1d83967dbec08c717e26cb70ca3ab123f372148cb2da3c662df8c5362a9a not found: ID does not exist"
Jul 28 20:29:35.962867 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.962842    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:29:35.970773 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:35.970751    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-635813-openshift-default-789ddbb6fb-28dlw"]
Jul 28 20:29:36.697197 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:36.697169    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" path="/var/lib/kubelet/pods/74390e16-83ca-4823-86f9-ee4f0f9e5a79/volumes"
Jul 28 20:29:37.064450 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.064425    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:37.210101 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.210068    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca\") pod \"40e50461-4adb-410a-a829-d0b5c645897f\" (UID: \"40e50461-4adb-410a-a829-d0b5c645897f\") "
Jul 28 20:29:37.210452 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.210433    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40e50461-4adb-410a-a829-d0b5c645897f" (UID: "40e50461-4adb-410a-a829-d0b5c645897f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:37.311169 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.311097    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40e50461-4adb-410a-a829-d0b5c645897f-maas-api-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:37.938186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.938154    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"
Jul 28 20:29:37.938338 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.938149    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n" event={"ID":"40e50461-4adb-410a-a829-d0b5c645897f","Type":"ContainerDied","Data":"dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c"}
Jul 28 20:29:37.938338 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:37.938277    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd496ba4c78444cffb77298fce52deb75ddbf26e46da18f6c6f6d0d7a35d313c"
Jul 28 20:29:49.709549 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:49.709513    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:29:49.710056 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:49.709741    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" podUID="ac1779bf-08cc-4a92-bf41-7341141416c9" containerName="maas-api" containerID="cri-o://794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b" gracePeriod=30
Jul 28 20:29:49.965379 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:49.965308    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:29:49.970453 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:49.970422    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"]
Jul 28 20:29:49.972163 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:49.972139    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7a9b9-bbl4n"]
Jul 28 20:29:50.018114 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.018074    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbkmp\" (UniqueName: \"kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp\") pod \"ac1779bf-08cc-4a92-bf41-7341141416c9\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") "
Jul 28 20:29:50.018314 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.018179    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") pod \"ac1779bf-08cc-4a92-bf41-7341141416c9\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") "
Jul 28 20:29:50.018314 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.018219    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca\") pod \"ac1779bf-08cc-4a92-bf41-7341141416c9\" (UID: \"ac1779bf-08cc-4a92-bf41-7341141416c9\") "
Jul 28 20:29:50.019046 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.019015    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac1779bf-08cc-4a92-bf41-7341141416c9" (UID: "ac1779bf-08cc-4a92-bf41-7341141416c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:50.022041 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.021903    2575 generic.go:356] "Generic (PLEG): container finished" podID="ac1779bf-08cc-4a92-bf41-7341141416c9" containerID="794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b" exitCode=0
Jul 28 20:29:50.022041 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.021983    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"
Jul 28 20:29:50.022779 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.021985    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" event={"ID":"ac1779bf-08cc-4a92-bf41-7341141416c9","Type":"ContainerDied","Data":"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"}
Jul 28 20:29:50.022779 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.022339    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5" event={"ID":"ac1779bf-08cc-4a92-bf41-7341141416c9","Type":"ContainerDied","Data":"ebd0ae9011b0fc62ac88e00f64406deec321cc1fdfde4de368570f46c6d4207d"}
Jul 28 20:29:50.022779 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.022403    2575 scope.go:117] "RemoveContainer" containerID="794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"
Jul 28 20:29:50.023373 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.023312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac1779bf-08cc-4a92-bf41-7341141416c9" (UID: "ac1779bf-08cc-4a92-bf41-7341141416c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:29:50.023975 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.023797    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp" (OuterVolumeSpecName: "kube-api-access-tbkmp") pod "ac1779bf-08cc-4a92-bf41-7341141416c9" (UID: "ac1779bf-08cc-4a92-bf41-7341141416c9"). InnerVolumeSpecName "kube-api-access-tbkmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:50.042072 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.042042    2575 scope.go:117] "RemoveContainer" containerID="794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"
Jul 28 20:29:50.042420 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:29:50.042395    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b\": container with ID starting with 794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b not found: ID does not exist" containerID="794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"
Jul 28 20:29:50.042487 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.042429    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b"} err="failed to get container status \"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b\": rpc error: code = NotFound desc = could not find container \"794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b\": container with ID starting with 794ccbb4febdfa224739a6cc00e8d62bd7206cd1496e37b346c8f9f793c18a8b not found: ID does not exist"
Jul 28 20:29:50.120153 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.120114    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbkmp\" (UniqueName: \"kubernetes.io/projected/ac1779bf-08cc-4a92-bf41-7341141416c9-kube-api-access-tbkmp\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.120153 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.120147    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac1779bf-08cc-4a92-bf41-7341141416c9-maas-api-tls\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.120153 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.120157    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac1779bf-08cc-4a92-bf41-7341141416c9-openshift-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.343442 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.343411    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:29:50.347178 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.347151    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d7a9b9-79fcbc7c96-xqhq5"]
Jul 28 20:29:50.696477 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.696399    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e50461-4adb-410a-a829-d0b5c645897f" path="/var/lib/kubelet/pods/40e50461-4adb-410a-a829-d0b5c645897f/volumes"
Jul 28 20:29:50.696900 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:29:50.696885    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac1779bf-08cc-4a92-bf41-7341141416c9" path="/var/lib/kubelet/pods/ac1779bf-08cc-4a92-bf41-7341141416c9/volumes"
Jul 28 20:30:00.247748 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.247690    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"]
Jul 28 20:30:00.248388 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248366    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.248448 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248391    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.248448 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248420    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.248448 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248428    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.248547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248452    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac1779bf-08cc-4a92-bf41-7341141416c9" containerName="maas-api"
Jul 28 20:30:00.248547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248462    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac1779bf-08cc-4a92-bf41-7341141416c9" containerName="maas-api"
Jul 28 20:30:00.248547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248471    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e50461-4adb-410a-a829-d0b5c645897f" containerName="revoke-keys"
Jul 28 20:30:00.248547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248480    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e50461-4adb-410a-a829-d0b5c645897f" containerName="revoke-keys"
Jul 28 20:30:00.248668 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248571    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.248668 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248585    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac1779bf-08cc-4a92-bf41-7341141416c9" containerName="maas-api"
Jul 28 20:30:00.248668 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248595    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e50461-4adb-410a-a829-d0b5c645897f" containerName="revoke-keys"
Jul 28 20:30:00.248668 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.248603    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="74390e16-83ca-4823-86f9-ee4f0f9e5a79" containerName="istio-proxy"
Jul 28 20:30:00.251872 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.251854    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"]
Jul 28 20:30:00.251990 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.251972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"
Jul 28 20:30:00.255267 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.255249    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"
Jul 28 20:30:00.257327 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.257307    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:30:00.260625 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.260605    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"]
Jul 28 20:30:00.263498 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.263480    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"
Jul 28 20:30:00.263859 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.263846    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"
Jul 28 20:30:00.271331 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.271310    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"
Jul 28 20:30:00.277371 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.275159    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"]
Jul 28 20:30:00.291837 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.291805    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"]
Jul 28 20:30:00.292345 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.292317    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"]
Jul 28 20:30:00.299029 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.299002    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"
Jul 28 20:30:00.312087 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.310955    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"]
Jul 28 20:30:00.314874 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.314777    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"]
Jul 28 20:30:00.317547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.317489    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"
Jul 28 20:30:00.329564 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.329508    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"
Jul 28 20:30:00.537795 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.537764    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"]
Jul 28 20:30:00.537947 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:30:00.537926    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06668a0f_46ed_4317_a2b1_30229ba9f0e5.slice/crio-b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6 WatchSource:0}: Error finding container b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6: Status 404 returned error can't find the container with id b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6
Jul 28 20:30:00.675037 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.675014    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"]
Jul 28 20:30:00.676405 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:30:00.676385    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f210c2d_8612_4cb3_bd08_7b84fc204442.slice/crio-7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e WatchSource:0}: Error finding container 7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e: Status 404 returned error can't find the container with id 7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e
Jul 28 20:30:00.746510 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.746475    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"]
Jul 28 20:30:00.750689 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:00.750657    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"]
Jul 28 20:30:00.751841 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:30:00.751818    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac130e55_8d5b_444a_a4e6_b44d7335c2b2.slice/crio-e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0 WatchSource:0}: Error finding container e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0: Status 404 returned error can't find the container with id e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0
Jul 28 20:30:00.752426 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:30:00.752403    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34db6ae_799f_48e7_91dc_21d48b4d336f.slice/crio-56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86 WatchSource:0}: Error finding container 56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86: Status 404 returned error can't find the container with id 56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86
Jul 28 20:30:01.083418 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.083385    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw" event={"ID":"3f210c2d-8612-4cb3-bd08-7b84fc204442","Type":"ContainerStarted","Data":"f245e25e5fc9cd2c3538dd4055b2a9ccad64baf86679ea9f4534fb4f734b8564"}
Jul 28 20:30:01.083547 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.083480    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw" event={"ID":"3f210c2d-8612-4cb3-bd08-7b84fc204442","Type":"ContainerStarted","Data":"7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e"}
Jul 28 20:30:01.084870 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.084841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md" event={"ID":"b34db6ae-799f-48e7-91dc-21d48b4d336f","Type":"ContainerStarted","Data":"56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86"}
Jul 28 20:30:01.086277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.086254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj" event={"ID":"ac130e55-8d5b-444a-a4e6-b44d7335c2b2","Type":"ContainerStarted","Data":"e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0"}
Jul 28 20:30:01.087719 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.087687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg" event={"ID":"06668a0f-46ed-4317-a2b1-30229ba9f0e5","Type":"ContainerStarted","Data":"2663d7e1c149f138a889f082b5a80662bdadd1199629666b8cd3b32d4f51388c"}
Jul 28 20:30:01.087719 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.087771    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg" event={"ID":"06668a0f-46ed-4317-a2b1-30229ba9f0e5","Type":"ContainerStarted","Data":"b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6"}
Jul 28 20:30:01.128102 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.128057    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg" podStartSLOduration=1.128042432 podStartE2EDuration="1.128042432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:01.123982273 +0000 UTC m=+3285.012859244" watchObservedRunningTime="2026-07-28 20:30:01.128042432 +0000 UTC m=+3285.016919402"
Jul 28 20:30:01.128360 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.128341    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw" podStartSLOduration=1.128336202 podStartE2EDuration="1.128336202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:01.101922087 +0000 UTC m=+3284.990799060" watchObservedRunningTime="2026-07-28 20:30:01.128336202 +0000 UTC m=+3285.017213173"
Jul 28 20:30:01.150634 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.150585    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md" podStartSLOduration=1.1505682560000001 podStartE2EDuration="1.150568256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:01.148980692 +0000 UTC m=+3285.037857662" watchObservedRunningTime="2026-07-28 20:30:01.150568256 +0000 UTC m=+3285.039445228"
Jul 28 20:30:01.171437 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:01.171399    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj" podStartSLOduration=1.171383918 podStartE2EDuration="1.171383918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:01.169597383 +0000 UTC m=+3285.058474355" watchObservedRunningTime="2026-07-28 20:30:01.171383918 +0000 UTC m=+3285.060260893"
Jul 28 20:30:02.095838 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.095803    2575 generic.go:356] "Generic (PLEG): container finished" podID="b34db6ae-799f-48e7-91dc-21d48b4d336f" containerID="797e2340e759dbaf91aa4c2c7e7d12179f79165a673910d6182a7ee9ce684790" exitCode=0
Jul 28 20:30:02.096260 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.095896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md" event={"ID":"b34db6ae-799f-48e7-91dc-21d48b4d336f","Type":"ContainerDied","Data":"797e2340e759dbaf91aa4c2c7e7d12179f79165a673910d6182a7ee9ce684790"}
Jul 28 20:30:02.097502 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.097480    2575 generic.go:356] "Generic (PLEG): container finished" podID="ac130e55-8d5b-444a-a4e6-b44d7335c2b2" containerID="cb1e9b97a270c04d8e43ba5538a1daaae2213613a6ff56f486765d2cd029b058" exitCode=0
Jul 28 20:30:02.097628 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.097565    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj" event={"ID":"ac130e55-8d5b-444a-a4e6-b44d7335c2b2","Type":"ContainerDied","Data":"cb1e9b97a270c04d8e43ba5538a1daaae2213613a6ff56f486765d2cd029b058"}
Jul 28 20:30:02.099074 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.099055    2575 generic.go:356] "Generic (PLEG): container finished" podID="06668a0f-46ed-4317-a2b1-30229ba9f0e5" containerID="2663d7e1c149f138a889f082b5a80662bdadd1199629666b8cd3b32d4f51388c" exitCode=0
Jul 28 20:30:02.099175 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.099079    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg" event={"ID":"06668a0f-46ed-4317-a2b1-30229ba9f0e5","Type":"ContainerDied","Data":"2663d7e1c149f138a889f082b5a80662bdadd1199629666b8cd3b32d4f51388c"}
Jul 28 20:30:02.100706 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.100687    2575 generic.go:356] "Generic (PLEG): container finished" podID="3f210c2d-8612-4cb3-bd08-7b84fc204442" containerID="f245e25e5fc9cd2c3538dd4055b2a9ccad64baf86679ea9f4534fb4f734b8564" exitCode=0
Jul 28 20:30:02.100826 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:02.100712    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw" event={"ID":"3f210c2d-8612-4cb3-bd08-7b84fc204442","Type":"ContainerDied","Data":"f245e25e5fc9cd2c3538dd4055b2a9ccad64baf86679ea9f4534fb4f734b8564"}
Jul 28 20:30:03.322659 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:03.322634    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"
Jul 28 20:30:03.324868 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:03.324850    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"
Jul 28 20:30:03.328445 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:03.328425    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"
Jul 28 20:30:03.331697 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:03.331680    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"
Jul 28 20:30:04.112534 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.112504    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md"
Jul 28 20:30:04.112748 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.112543    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-dl8md" event={"ID":"b34db6ae-799f-48e7-91dc-21d48b4d336f","Type":"ContainerDied","Data":"56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86"}
Jul 28 20:30:04.112748 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.112587    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56fec09e91fadb2b99258b26b7462e7be881aa345761c81af1d40c3e580b2d86"
Jul 28 20:30:04.113843 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.113823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"
Jul 28 20:30:04.113843 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.113834    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj" event={"ID":"ac130e55-8d5b-444a-a4e6-b44d7335c2b2","Type":"ContainerDied","Data":"e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0"}
Jul 28 20:30:04.114037 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.113876    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e70115b99a2e26b37e7f97f0796710849d04c2014a29de042064a96886f90fc0"
Jul 28 20:30:04.115059 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.115041    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"
Jul 28 20:30:04.115191 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.115077    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg" event={"ID":"06668a0f-46ed-4317-a2b1-30229ba9f0e5","Type":"ContainerDied","Data":"b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6"}
Jul 28 20:30:04.115191 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.115116    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2eb3ae32fc2ca7264fe9412e8ef2c3dbbf506dd8c5cd66ae41f950f6cbe95d6"
Jul 28 20:30:04.116405 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.116380    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw" event={"ID":"3f210c2d-8612-4cb3-bd08-7b84fc204442","Type":"ContainerDied","Data":"7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e"}
Jul 28 20:30:04.116405 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.116399    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"
Jul 28 20:30:04.116589 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.116425    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f826f7d2fda0da6ef0b67f77f89ca6c4b6d2cff86e30523b5e2140fba95755e"
Jul 28 20:30:04.432307 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.432222    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"]
Jul 28 20:30:04.445124 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.445098    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-rwspv"]
Jul 28 20:30:04.698413 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:04.698320    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdb20187-f48a-4dc4-bd63-e51fad11f836" path="/var/lib/kubelet/pods/fdb20187-f48a-4dc4-bd63-e51fad11f836/volumes"
Jul 28 20:30:16.814594 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:16.814567    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:30:16.823067 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:16.823044    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m9nxc_bfe093a0-5621-47a9-b0eb-1443a018c1f2/ovn-acl-logging/0.log"
Jul 28 20:30:17.684921 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:30:17.684888    2575 scope.go:117] "RemoveContainer" containerID="e3f4f7613ddb047b376a9a699ff9d0dac26ae264613287bbe050102fc6267b26"
Jul 28 20:31:05.383401 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.383366    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"]
Jul 28 20:31:05.384052 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384031    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f210c2d-8612-4cb3-bd08-7b84fc204442" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384054    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f210c2d-8612-4cb3-bd08-7b84fc204442" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384084    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34db6ae-799f-48e7-91dc-21d48b4d336f" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384093    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34db6ae-799f-48e7-91dc-21d48b4d336f" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384121    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06668a0f-46ed-4317-a2b1-30229ba9f0e5" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384129    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="06668a0f-46ed-4317-a2b1-30229ba9f0e5" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384141    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac130e55-8d5b-444a-a4e6-b44d7335c2b2" containerName="cleanup"
Jul 28 20:31:05.384154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384149    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac130e55-8d5b-444a-a4e6-b44d7335c2b2" containerName="cleanup"
Jul 28 20:31:05.384490 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384263    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac130e55-8d5b-444a-a4e6-b44d7335c2b2" containerName="cleanup"
Jul 28 20:31:05.384490 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384278    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="06668a0f-46ed-4317-a2b1-30229ba9f0e5" containerName="cleanup"
Jul 28 20:31:05.384490 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384293    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34db6ae-799f-48e7-91dc-21d48b4d336f" containerName="cleanup"
Jul 28 20:31:05.384490 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.384303    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f210c2d-8612-4cb3-bd08-7b84fc204442" containerName="cleanup"
Jul 28 20:31:05.387757 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.387717    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:05.390296 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.390272    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:31:05.396890 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.396854    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"]
Jul 28 20:31:05.498030 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.497999    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw\" (UID: \"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:05.598954 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.598919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw\" (UID: \"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:05.599591 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.599566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw\" (UID: \"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:05.703003 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.702922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:05.848107 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:05.848082    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"]
Jul 28 20:31:05.850008 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:31:05.849981    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51de581c_a8e0_40f8_b9c1_01ddc8bfe92b.slice/crio-eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239 WatchSource:0}: Error finding container eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239: Status 404 returned error can't find the container with id eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239
Jul 28 20:31:06.450477 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:06.450386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw" event={"ID":"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b","Type":"ContainerStarted","Data":"edac0f0e4937a1cbf67e4b78d3cca03119e165c6adfc346c18ce129ac78345a4"}
Jul 28 20:31:06.450477 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:06.450442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw" event={"ID":"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b","Type":"ContainerStarted","Data":"eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239"}
Jul 28 20:31:06.467655 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:06.467595    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw" podStartSLOduration=1.467576184 podStartE2EDuration="1.467576184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:06.466818233 +0000 UTC m=+3350.355695206" watchObservedRunningTime="2026-07-28 20:31:06.467576184 +0000 UTC m=+3350.356453157"
Jul 28 20:31:07.457081 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:07.457049    2575 generic.go:356] "Generic (PLEG): container finished" podID="51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" containerID="edac0f0e4937a1cbf67e4b78d3cca03119e165c6adfc346c18ce129ac78345a4" exitCode=0
Jul 28 20:31:07.457492 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:07.457104    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw" event={"ID":"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b","Type":"ContainerDied","Data":"edac0f0e4937a1cbf67e4b78d3cca03119e165c6adfc346c18ce129ac78345a4"}
Jul 28 20:31:08.600041 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:08.600016    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:08.731323 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:08.731232    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca\") pod \"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b\" (UID: \"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b\") "
Jul 28 20:31:08.731582 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:08.731559    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" (UID: "51de581c-a8e0-40f8-b9c1-01ddc8bfe92b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:08.832382 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:08.832341    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b-maas-api-service-ca\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:31:09.469107 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:09.469062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw" event={"ID":"51de581c-a8e0-40f8-b9c1-01ddc8bfe92b","Type":"ContainerDied","Data":"eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239"}
Jul 28 20:31:09.469107 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:09.469090    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"
Jul 28 20:31:09.469383 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:09.469119    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eecc7ef0527a687c79969c32d7427836be6a25b1efa66218e42a41ab6a16a239"
Jul 28 20:31:16.145113 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.145073    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"]
Jul 28 20:31:16.150933 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.150901    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-f11e93-29754510-6krwg"]
Jul 28 20:31:16.209461 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.209423    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:31:16.209756 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.209686    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" podUID="ec5121c9-6490-439f-8085-a759953c399f" containerName="payload-pre-processing" containerID="cri-o://4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff" gracePeriod=30
Jul 28 20:31:16.442985 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.442882    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"]
Jul 28 20:31:16.445399 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.445376    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f11e93-226lw"]
Jul 28 20:31:16.484004 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.483979    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:31:16.508078 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.508049    2575 generic.go:356] "Generic (PLEG): container finished" podID="ec5121c9-6490-439f-8085-a759953c399f" containerID="4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff" exitCode=0
Jul 28 20:31:16.508213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.508089    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"
Jul 28 20:31:16.508213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.508125    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" event={"ID":"ec5121c9-6490-439f-8085-a759953c399f","Type":"ContainerDied","Data":"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"}
Jul 28 20:31:16.508213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.508180    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7" event={"ID":"ec5121c9-6490-439f-8085-a759953c399f","Type":"ContainerDied","Data":"a97bb8a1a410ad9ca51dfd8b8d8d131880ad55385fabe3303d70b3a36cb04930"}
Jul 28 20:31:16.508213 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.508209    2575 scope.go:117] "RemoveContainer" containerID="4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"
Jul 28 20:31:16.517658 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.517642    2575 scope.go:117] "RemoveContainer" containerID="4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"
Jul 28 20:31:16.517965 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:31:16.517941    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff\": container with ID starting with 4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff not found: ID does not exist" containerID="4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"
Jul 28 20:31:16.518022 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.517980    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff"} err="failed to get container status \"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff\": rpc error: code = NotFound desc = could not find container \"4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff\": container with ID starting with 4137d5d3e6fabe4a7a15058cc9b805c11d98155c05cea61ee5d573a78f8ef5ff not found: ID does not exist"
Jul 28 20:31:16.610315 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.610284    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume\") pod \"ec5121c9-6490-439f-8085-a759953c399f\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") "
Jul 28 20:31:16.610461 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.610334    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b727s\" (UniqueName: \"kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s\") pod \"ec5121c9-6490-439f-8085-a759953c399f\" (UID: \"ec5121c9-6490-439f-8085-a759953c399f\") "
Jul 28 20:31:16.610647 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.610620    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec5121c9-6490-439f-8085-a759953c399f" (UID: "ec5121c9-6490-439f-8085-a759953c399f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:16.612479 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.612457    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s" (OuterVolumeSpecName: "kube-api-access-b727s") pod "ec5121c9-6490-439f-8085-a759953c399f" (UID: "ec5121c9-6490-439f-8085-a759953c399f"). InnerVolumeSpecName "kube-api-access-b727s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:16.695850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.695779    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06668a0f-46ed-4317-a2b1-30229ba9f0e5" path="/var/lib/kubelet/pods/06668a0f-46ed-4317-a2b1-30229ba9f0e5/volumes"
Jul 28 20:31:16.696071 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.696058    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" path="/var/lib/kubelet/pods/51de581c-a8e0-40f8-b9c1-01ddc8bfe92b/volumes"
Jul 28 20:31:16.711336 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.711316    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec5121c9-6490-439f-8085-a759953c399f-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.711336 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.711336    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b727s\" (UniqueName: \"kubernetes.io/projected/ec5121c9-6490-439f-8085-a759953c399f-kube-api-access-b727s\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.829484 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.829450    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:31:16.839403 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:16.839374    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f11e93-d69bd7cb8-4zwr7"]
Jul 28 20:31:18.695276 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:18.695240    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec5121c9-6490-439f-8085-a759953c399f" path="/var/lib/kubelet/pods/ec5121c9-6490-439f-8085-a759953c399f/volumes"
Jul 28 20:31:35.085111 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:35.085074    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"]
Jul 28 20:31:35.094450 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:35.094417    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bd928b-29754510-lvkqj"]
Jul 28 20:31:36.695356 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:36.695322    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac130e55-8d5b-444a-a4e6-b44d7335c2b2" path="/var/lib/kubelet/pods/ac130e55-8d5b-444a-a4e6-b44d7335c2b2/volumes"
Jul 28 20:31:55.033952 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.033919    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"]
Jul 28 20:31:55.040364 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.040329    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-60efb4-29754510-pzwdw"]
Jul 28 20:31:55.084235 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.084200    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:31:55.084434 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.084411    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" podUID="178fd2e1-279f-4936-abee-0452a19bfdda" containerName="payload-pre-processing" containerID="cri-o://d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39" gracePeriod=30
Jul 28 20:31:55.352252 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.352072    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:31:55.460125 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.460098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hhzl\" (UniqueName: \"kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl\") pod \"178fd2e1-279f-4936-abee-0452a19bfdda\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") "
Jul 28 20:31:55.460125 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.460129    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume\") pod \"178fd2e1-279f-4936-abee-0452a19bfdda\" (UID: \"178fd2e1-279f-4936-abee-0452a19bfdda\") "
Jul 28 20:31:55.460538 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.460515    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "178fd2e1-279f-4936-abee-0452a19bfdda" (UID: "178fd2e1-279f-4936-abee-0452a19bfdda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:55.462033 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.462010    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl" (OuterVolumeSpecName: "kube-api-access-7hhzl") pod "178fd2e1-279f-4936-abee-0452a19bfdda" (UID: "178fd2e1-279f-4936-abee-0452a19bfdda"). InnerVolumeSpecName "kube-api-access-7hhzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:55.561674 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.561644    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hhzl\" (UniqueName: \"kubernetes.io/projected/178fd2e1-279f-4936-abee-0452a19bfdda-kube-api-access-7hhzl\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.561674 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.561668    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178fd2e1-279f-4936-abee-0452a19bfdda-plugins-config-volume\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.720207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.720116    2575 generic.go:356] "Generic (PLEG): container finished" podID="178fd2e1-279f-4936-abee-0452a19bfdda" containerID="d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39" exitCode=0
Jul 28 20:31:55.720207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.720161    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"
Jul 28 20:31:55.720207 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.720166    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" event={"ID":"178fd2e1-279f-4936-abee-0452a19bfdda","Type":"ContainerDied","Data":"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"}
Jul 28 20:31:55.720481 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.720234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv" event={"ID":"178fd2e1-279f-4936-abee-0452a19bfdda","Type":"ContainerDied","Data":"48574ff59a44caea752f3a1b9a03413e9e58a2ce4a84752829e343f05b32f7c9"}
Jul 28 20:31:55.720481 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.720259    2575 scope.go:117] "RemoveContainer" containerID="d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"
Jul 28 20:31:55.729934 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.729919    2575 scope.go:117] "RemoveContainer" containerID="d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"
Jul 28 20:31:55.730183 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:31:55.730165    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39\": container with ID starting with d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39 not found: ID does not exist" containerID="d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"
Jul 28 20:31:55.730229 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.730192    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39"} err="failed to get container status \"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39\": rpc error: code = NotFound desc = could not find container \"d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39\": container with ID starting with d8b74bfe0219e8b1c7fed59cbb67c2661505addae428ac2aaf5f8293fd66ed39 not found: ID does not exist"
Jul 28 20:31:55.865775 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.865721    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:31:55.911032 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:55.910988    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-60efb4-779fb6cccb-x4mqv"]
Jul 28 20:31:56.695584 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:56.695543    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="178fd2e1-279f-4936-abee-0452a19bfdda" path="/var/lib/kubelet/pods/178fd2e1-279f-4936-abee-0452a19bfdda/volumes"
Jul 28 20:31:56.696125 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:31:56.696081    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f210c2d-8612-4cb3-bd08-7b84fc204442" path="/var/lib/kubelet/pods/3f210c2d-8612-4cb3-bd08-7b84fc204442/volumes"
Jul 28 20:32:02.177013 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:02.176979    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:32:02.177408 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:02.177212    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy" containerID="cri-o://f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82" gracePeriod=30
Jul 28 20:32:07.442251 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.442191    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:32:07.572674 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572648    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.572850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572691    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.572850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572725    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.572938 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572917    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.572991 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.573055 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.572990    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.573055 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573010    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:07.573055 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573050    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.573215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573104    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.573215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573128    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkfjd\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd\") pod \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\" (UID: \"a4b6aad6-8ebe-4fc3-aea9-883214823c5c\") "
Jul 28 20:32:07.573215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:07.573386 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:07.573386 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573356    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:32:07.573501 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573484    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-certs\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.573539 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573508    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-credential-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.573539 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573523    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istiod-ca-cert\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.573539 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573535    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-workload-socket\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.574007 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.573982    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:07.574972 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.574950    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:07.575045 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.574970    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:07.575450 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.575428    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd" (OuterVolumeSpecName: "kube-api-access-bkfjd") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "kube-api-access-bkfjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:07.575563 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.575549    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4b6aad6-8ebe-4fc3-aea9-883214823c5c" (UID: "a4b6aad6-8ebe-4fc3-aea9-883214823c5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 20:32:07.674753 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.674710    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-token\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.674888 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.674758    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-envoy\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.674888 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.674774    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-podinfo\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.674888 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.674786    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-istio-data\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.674888 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.674799    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkfjd\" (UniqueName: \"kubernetes.io/projected/a4b6aad6-8ebe-4fc3-aea9-883214823c5c-kube-api-access-bkfjd\") on node \"ip-10-0-130-70.ec2.internal\" DevicePath \"\""
Jul 28 20:32:07.784097 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.784029    2575 generic.go:356] "Generic (PLEG): container finished" podID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerID="f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82" exitCode=0
Jul 28 20:32:07.784097 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.784081    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"
Jul 28 20:32:07.784097 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.784092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerDied","Data":"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"}
Jul 28 20:32:07.784486 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.784134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n" event={"ID":"a4b6aad6-8ebe-4fc3-aea9-883214823c5c","Type":"ContainerDied","Data":"51699ade76c4a6f2d2a951f59f35edf860c191643604215f4b9c14c035f53972"}
Jul 28 20:32:07.784486 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.784157    2575 scope.go:117] "RemoveContainer" containerID="f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"
Jul 28 20:32:07.793392 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.793373    2575 scope.go:117] "RemoveContainer" containerID="f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"
Jul 28 20:32:07.801115 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.801100    2575 scope.go:117] "RemoveContainer" containerID="f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"
Jul 28 20:32:07.801366 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:32:07.801340    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82\": container with ID starting with f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82 not found: ID does not exist" containerID="f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"
Jul 28 20:32:07.801413 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.801369    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82"} err="failed to get container status \"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82\": rpc error: code = NotFound desc = could not find container \"f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82\": container with ID starting with f2e53db33e1170245b5a12efbf86dcfa7803a535ee9a07d661b637d3b210bc82 not found: ID does not exist"
Jul 28 20:32:07.801413 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.801387    2575 scope.go:117] "RemoveContainer" containerID="f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"
Jul 28 20:32:07.801623 ip-10-0-130-70 kubenswrapper[2575]: E0728 20:32:07.801605    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865\": container with ID starting with f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865 not found: ID does not exist" containerID="f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"
Jul 28 20:32:07.801661 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.801630    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865"} err="failed to get container status \"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865\": rpc error: code = NotFound desc = could not find container \"f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865\": container with ID starting with f58f39c39f3c35b47cbe933be5f0186aed6960a132d82624b9a64afe34363865 not found: ID does not exist"
Jul 28 20:32:07.808198 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.808173    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:32:07.812128 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:07.812107    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-60efb4-openshift-default-f8c7dfff6-svt9n"]
Jul 28 20:32:08.695253 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:08.695219    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" path="/var/lib/kubelet/pods/a4b6aad6-8ebe-4fc3-aea9-883214823c5c/volumes"
Jul 28 20:32:08.940434 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:08.940399    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-54bcf7858d-kv7nq_6eb12912-cfde-4981-8ff2-d35874285a88/manager/0.log"
Jul 28 20:32:09.213164 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:09.213134    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5p84w_79098d57-0b49-4fad-951f-891a2ee5d42c/manager/0.log"
Jul 28 20:32:09.490679 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:09.490595    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-85b9t_41dd91d3-1775-4fd6-bb7a-e746ab69fbd5/manager/0.log"
Jul 28 20:32:09.504121 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:09.504098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-85b9t_41dd91d3-1775-4fd6-bb7a-e746ab69fbd5/copy-manifests/0.log"
Jul 28 20:32:10.046710 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:10.046672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5f8bcdcb76-htjrq_077d0a51-2f26-4f99-a2de-aa355e4cd1cd/maas-api/0.log"
Jul 28 20:32:10.422491 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:10.422460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754510-dl8md_b34db6ae-799f-48e7-91dc-21d48b4d336f/cleanup/0.log"
Jul 28 20:32:10.551247 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:10.551216    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6rwkl_b2a2923b-d563-47cc-88b1-b86ce977f6d0/postgres/0.log"
Jul 28 20:32:12.118793 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:12.118750    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-btsfx_df0b661f-828e-4b83-898d-31ab134e1728/manager/0.log"
Jul 28 20:32:12.242996 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:12.242963    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-npsd5_4e6af6e7-74a7-41c4-ba25-c45a4545845a/manager/0.log"
Jul 28 20:32:12.496325 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:12.496245    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2r5b9_98a87f4d-2bda-4cd6-a724-cfa574672365/registry-server/0.log"
Jul 28 20:32:12.641779 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:12.641722    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g9bbb_99f01c69-5d89-41fb-ac7b-8bbb37ebef69/manager/0.log"
Jul 28 20:32:12.911419 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:12.911383    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gpqr2_d19913b3-f734-4b87-b076-524d7d484f8a/manager/0.log"
Jul 28 20:32:13.450186 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:13.450151    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2n7lp_9359fcb2-2da7-4c37-8ad6-c998f2830dbf/discovery/0.log"
Jul 28 20:32:13.699619 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:13.699585    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f9868ff69-zw7pq_ff98df05-5129-4bcb-b8b9-35db618ea6e6/kube-auth-proxy/0.log"
Jul 28 20:32:13.834936 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:13.834905    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/1.log"
Jul 28 20:32:13.974961 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:13.974917    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dfhb5_2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0/payload-pre-processing/0.log"
Jul 28 20:32:14.113510 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:14.113419    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"
Jul 28 20:32:14.620701 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:14.620665    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nwk7j_3262c41b-b5dc-4ce1-9b87-0afca7193c12/main/0.log"
Jul 28 20:32:14.864083 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:14.864055    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2jxmm_3b3b3ed7-feb7-420d-b5a4-6defa375e139/main/0.log"
Jul 28 20:32:24.099319 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:24.099285    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cgvwx_b7d86f9a-aae2-46d4-b6d3-6831fea265bc/global-pull-secret-syncer/0.log"
Jul 28 20:32:24.237490 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:24.237459    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7x85f_f8767354-38e8-48ec-9349-3ea7db7c0258/konnectivity-agent/0.log"
Jul 28 20:32:24.380798 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:24.380709    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-70.ec2.internal_e33f5784a8667eeea92ef51af7a40e20/haproxy/0.log"
Jul 28 20:32:29.559011 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:29.558982    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-btsfx_df0b661f-828e-4b83-898d-31ab134e1728/manager/0.log"
Jul 28 20:32:29.593163 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:29.593120    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-npsd5_4e6af6e7-74a7-41c4-ba25-c45a4545845a/manager/0.log"
Jul 28 20:32:29.687747 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:29.687647    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2r5b9_98a87f4d-2bda-4cd6-a724-cfa574672365/registry-server/0.log"
Jul 28 20:32:29.916895 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:29.916862    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g9bbb_99f01c69-5d89-41fb-ac7b-8bbb37ebef69/manager/0.log"
Jul 28 20:32:30.059025 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:30.058994    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gpqr2_d19913b3-f734-4b87-b076-524d7d484f8a/manager/0.log"
Jul 28 20:32:31.920289 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:31.920253    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fzm8r_e2d88fce-2d42-4c48-81ac-b615150d64fb/kube-state-metrics/0.log"
Jul 28 20:32:31.943576 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:31.943550    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fzm8r_e2d88fce-2d42-4c48-81ac-b615150d64fb/kube-rbac-proxy-main/0.log"
Jul 28 20:32:31.965743 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:31.965706    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fzm8r_e2d88fce-2d42-4c48-81ac-b615150d64fb/kube-rbac-proxy-self/0.log"
Jul 28 20:32:32.215410 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.215376    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxh9w_a7c4da5c-8150-4a47-a48e-28cc564d77a3/node-exporter/0.log"
Jul 28 20:32:32.235982 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.235959    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxh9w_a7c4da5c-8150-4a47-a48e-28cc564d77a3/kube-rbac-proxy/0.log"
Jul 28 20:32:32.259416 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.259393    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxh9w_a7c4da5c-8150-4a47-a48e-28cc564d77a3/init-textfile/0.log"
Jul 28 20:32:32.292399 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.292371    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fwqnv_8ff38f0a-2b30-4f78-baf3-c97620f1a92b/kube-rbac-proxy-main/0.log"
Jul 28 20:32:32.315193 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.315172    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fwqnv_8ff38f0a-2b30-4f78-baf3-c97620f1a92b/kube-rbac-proxy-self/0.log"
Jul 28 20:32:32.340162 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.340135    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-fwqnv_8ff38f0a-2b30-4f78-baf3-c97620f1a92b/openshift-state-metrics/0.log"
Jul 28 20:32:32.391944 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.391914    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/prometheus/0.log"
Jul 28 20:32:32.432554 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.432530    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/config-reloader/0.log"
Jul 28 20:32:32.462511 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.462482    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/thanos-sidecar/0.log"
Jul 28 20:32:32.489632 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.489570    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/kube-rbac-proxy-web/0.log"
Jul 28 20:32:32.515460 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.515433    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/kube-rbac-proxy/0.log"
Jul 28 20:32:32.562049 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.562023    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/kube-rbac-proxy-thanos/0.log"
Jul 28 20:32:32.595912 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.595888    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_80a50eb3-07fa-4875-a492-32f32387021e/init-config-reloader/0.log"
Jul 28 20:32:32.686598 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.686576    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-4whjh_e54e6bb1-2403-4917-9e45-822bcf490511/prometheus-operator-admission-webhook/0.log"
Jul 28 20:32:32.800877 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.800808    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/thanos-query/0.log"
Jul 28 20:32:32.821165 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.821143    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/kube-rbac-proxy-web/0.log"
Jul 28 20:32:32.842956 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.842926    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/kube-rbac-proxy/0.log"
Jul 28 20:32:32.863563 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.863545    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/prom-label-proxy/0.log"
Jul 28 20:32:32.886215 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.886182    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/kube-rbac-proxy-rules/0.log"
Jul 28 20:32:32.907525 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:32.907503    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7574b74685-zl65p_c2e31e3c-9995-4079-b634-b0dca2866336/kube-rbac-proxy-metrics/0.log"
Jul 28 20:32:33.508968 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.508932    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"]
Jul 28 20:32:33.509513 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509492    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="178fd2e1-279f-4936-abee-0452a19bfdda" containerName="payload-pre-processing"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509515    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="178fd2e1-279f-4936-abee-0452a19bfdda" containerName="payload-pre-processing"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509532    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" containerName="revoke-keys"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509540    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" containerName="revoke-keys"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509563    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509574    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509605 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509604    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509612    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509622    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec5121c9-6490-439f-8085-a759953c399f" containerName="payload-pre-processing"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509630    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec5121c9-6490-439f-8085-a759953c399f" containerName="payload-pre-processing"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509719    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509749    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="51de581c-a8e0-40f8-b9c1-01ddc8bfe92b" containerName="revoke-keys"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509762    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="178fd2e1-279f-4936-abee-0452a19bfdda" containerName="payload-pre-processing"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509774    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b6aad6-8ebe-4fc3-aea9-883214823c5c" containerName="istio-proxy"
Jul 28 20:32:33.509950 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.509786    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec5121c9-6490-439f-8085-a759953c399f" containerName="payload-pre-processing"
Jul 28 20:32:33.513098 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.513077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.515314 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.515291    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z498q\"/\"openshift-service-ca.crt\""
Jul 28 20:32:33.515405 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.515291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z498q\"/\"default-dockercfg-xg8mv\""
Jul 28 20:32:33.516030 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.516012    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z498q\"/\"kube-root-ca.crt\""
Jul 28 20:32:33.520698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.520677    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"]
Jul 28 20:32:33.603123 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.603093    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-sys\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.603123 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.603125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-podres\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.603321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.603153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k68p9\" (UniqueName: \"kubernetes.io/projected/bcb24b78-0592-457e-a86b-f133e8032b24-kube-api-access-k68p9\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.603321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.603181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-lib-modules\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.603321 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.603207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-proc\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.703850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-sys\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.703850 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-podres\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k68p9\" (UniqueName: \"kubernetes.io/projected/bcb24b78-0592-457e-a86b-f133e8032b24-kube-api-access-k68p9\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-lib-modules\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-sys\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-proc\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.703998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-podres\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704062 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.704027    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-lib-modules\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.704277 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.704079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bcb24b78-0592-457e-a86b-f133e8032b24-proc\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.713145 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.713123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k68p9\" (UniqueName: \"kubernetes.io/projected/bcb24b78-0592-457e-a86b-f133e8032b24-kube-api-access-k68p9\") pod \"perf-node-gather-daemonset-nwsxc\" (UID: \"bcb24b78-0592-457e-a86b-f133e8032b24\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.823929 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.823856    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:33.954672 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:33.954636    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"]
Jul 28 20:32:33.956876 ip-10-0-130-70 kubenswrapper[2575]: W0728 20:32:33.956843    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbcb24b78_0592_457e_a86b_f133e8032b24.slice/crio-0a7224f9b181b730ed8b2e00faeb313eb79fab0d408ae0a6000f6dae3b584970 WatchSource:0}: Error finding container 0a7224f9b181b730ed8b2e00faeb313eb79fab0d408ae0a6000f6dae3b584970: Status 404 returned error can't find the container with id 0a7224f9b181b730ed8b2e00faeb313eb79fab0d408ae0a6000f6dae3b584970
Jul 28 20:32:34.928267 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:34.928234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc" event={"ID":"bcb24b78-0592-457e-a86b-f133e8032b24","Type":"ContainerStarted","Data":"72d8d979a1eaf92b954e7ae5d7c7d0010068d2717eed4b4e7383e6893113ecf9"}
Jul 28 20:32:34.928793 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:34.928283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc" event={"ID":"bcb24b78-0592-457e-a86b-f133e8032b24","Type":"ContainerStarted","Data":"0a7224f9b181b730ed8b2e00faeb313eb79fab0d408ae0a6000f6dae3b584970"}
Jul 28 20:32:34.928793 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:34.928333    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:34.953348 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:34.953285    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc" podStartSLOduration=1.953263807 podStartE2EDuration="1.953263807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:34.945468504 +0000 UTC m=+3438.834345476" watchObservedRunningTime="2026-07-28 20:32:34.953263807 +0000 UTC m=+3438.842140781"
Jul 28 20:32:36.713842 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:36.713812    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czt8t_c96ef683-1e6b-4900-9520-84e87a7fe170/dns/0.log"
Jul 28 20:32:36.737819 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:36.737787    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czt8t_c96ef683-1e6b-4900-9520-84e87a7fe170/kube-rbac-proxy/0.log"
Jul 28 20:32:36.898867 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:36.898835    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q6pj8_8318056b-0056-4938-a330-db29a044c0d9/dns-node-resolver/0.log"
Jul 28 20:32:37.504799 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:37.504771    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8647557848-sq5kj_6e904d8f-e0f2-46c6-99fe-db4c5a883f59/registry/0.log"
Jul 28 20:32:37.555318 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:37.555273    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvt4g_fb9c0960-6f62-4fbb-838c-9f436afebb45/node-ca/0.log"
Jul 28 20:32:38.746940 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.746906    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2n7lp_9359fcb2-2da7-4c37-8ad6-c998f2830dbf/discovery/0.log"
Jul 28 20:32:38.793507 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.793477    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f9868ff69-zw7pq_ff98df05-5129-4bcb-b8b9-35db618ea6e6/kube-auth-proxy/0.log"
Jul 28 20:32:38.824265 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.824182    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/0.log"
Jul 28 20:32:38.885085 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.885054    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/1.log"
Jul 28 20:32:38.925875 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.925798    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dfhb5_2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0/payload-pre-processing/0.log"
Jul 28 20:32:38.964897 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:38.964870    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"
Jul 28 20:32:39.801021 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:39.800982    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2rblv_5369d78b-34e1-45f1-ac78-5260c5f7027b/serve-healthcheck-canary/0.log"
Jul 28 20:32:40.499567 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:40.499534    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfpc_72b1deed-76bb-4fa0-bacb-e4c8e915fabb/kube-rbac-proxy/0.log"
Jul 28 20:32:40.523042 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:40.523018    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfpc_72b1deed-76bb-4fa0-bacb-e4c8e915fabb/exporter/0.log"
Jul 28 20:32:40.545698 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:40.545667    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfpc_72b1deed-76bb-4fa0-bacb-e4c8e915fabb/extractor/0.log"
Jul 28 20:32:40.943521 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:40.943496    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-nwsxc"
Jul 28 20:32:43.933273 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:43.933219    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-54bcf7858d-kv7nq_6eb12912-cfde-4981-8ff2-d35874285a88/manager/0.log"
Jul 28 20:32:44.092465 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.092426    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5p84w_79098d57-0b49-4fad-951f-891a2ee5d42c/manager/0.log"
Jul 28 20:32:44.332745 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.332696    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-85b9t_41dd91d3-1775-4fd6-bb7a-e746ab69fbd5/manager/0.log"
Jul 28 20:32:44.343043 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.343016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2n7lp_9359fcb2-2da7-4c37-8ad6-c998f2830dbf/discovery/0.log"
Jul 28 20:32:44.357614 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.357592    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-85b9t_41dd91d3-1775-4fd6-bb7a-e746ab69fbd5/copy-manifests/0.log"
Jul 28 20:32:44.370199 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.370173    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f9868ff69-zw7pq_ff98df05-5129-4bcb-b8b9-35db618ea6e6/kube-auth-proxy/0.log"
Jul 28 20:32:44.392245 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.392187    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/0.log"
Jul 28 20:32:44.443064 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.443009    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/1.log"
Jul 28 20:32:44.479414 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.479390    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dfhb5_2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0/payload-pre-processing/0.log"
Jul 28 20:32:44.521534 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:44.521508    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"
Jul 28 20:32:46.289351 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.289320    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2n7lp_9359fcb2-2da7-4c37-8ad6-c998f2830dbf/discovery/0.log"
Jul 28 20:32:46.332410 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.332382    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f9868ff69-zw7pq_ff98df05-5129-4bcb-b8b9-35db618ea6e6/kube-auth-proxy/0.log"
Jul 28 20:32:46.354154 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.354130    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/0.log"
Jul 28 20:32:46.410609 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.410583    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lcz4q_efd39ec7-ad59-4d89-bab6-22dc6504b258/istio-proxy/1.log"
Jul 28 20:32:46.443198 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.443172    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dfhb5_2943ecc8-1f7d-460d-939d-7cf8f4ce8fc0/payload-pre-processing/0.log"
Jul 28 20:32:46.480777 ip-10-0-130-70 kubenswrapper[2575]: I0728 20:32:46.480751    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qqsrk_9ed95a3f-c063-4014-be4a-b1083e72e5b0/payload-processing/0.log"