Jul 28 19:00:12.085195 ip-10-0-142-99 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:00:12.085204 ip-10-0-142-99 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:00:12.085210 ip-10-0-142-99 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:00:12.085416 ip-10-0-142-99 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:00:22.106197 ip-10-0-142-99 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 19:00:22.106220 ip-10-0-142-99 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8cadf0a3199641c2b8c5f9137ff36523 --
Jul 28 19:02:11.415391 ip-10-0-142-99 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:02:11.729740 ip-10-0-142-99 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:11.729740 ip-10-0-142-99 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:02:11.729740 ip-10-0-142-99 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:11.729740 ip-10-0-142-99 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:02:11.729740 ip-10-0-142-99 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:11.731947 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.731837    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:02:11.735677 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735663    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.735677 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735678    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735682    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735685    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735691    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735695    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735699    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735702    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735705    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735708    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735711    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735714    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735716    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735719    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735722    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735725    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735727    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735730    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735733    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735735    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.735739 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735738    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735740    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735743    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735746    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735749    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735753    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735756    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735759    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735761    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735764    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735767    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735769    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735772    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735775    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735778    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735783    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735786    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735789    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735792    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735795    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.736225 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735797    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735800    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735803    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735806    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735808    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735811    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735813    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735816    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735819    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735821    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735824    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735827    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735830    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735832    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735836    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735838    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735841    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735844    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735846    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.736706 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735849    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735851    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735854    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735857    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735859    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735862    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735864    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735867    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735869    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735871    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735874    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735877    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735879    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735882    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735884    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735888    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735892    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735896    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735898    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735901    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.737219 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735903    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735906    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735909    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735911    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735913    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735916    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735919    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735922    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735925    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735927    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735930    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735932    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735935    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735937    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735940    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735943    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735945    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735948    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735951    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.735953    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.737692 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736719    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736726    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736729    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736732    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736735    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736737    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736740    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736743    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736746    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736749    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736751    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736754    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736757    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736759    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736762    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736765    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736768    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736770    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736773    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736776    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.738317 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736779    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736782    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736785    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736787    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736790    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736793    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736795    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736798    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736801    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736803    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736805    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736818    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736822    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736824    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736827    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736831    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736833    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736836    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736839    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736841    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.738809 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736844    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736848    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736851    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736854    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736856    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736859    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736864    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736867    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736870    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736873    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736876    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736879    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736882    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736885    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736888    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736891    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736893    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736896    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736899    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.739350 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736902    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736905    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736908    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736910    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736913    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736915    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736918    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736921    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736923    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736925    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736928    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736931    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736933    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736937    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736939    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736942    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736945    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736947    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736950    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.739810 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736952    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736955    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736957    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736960    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736962    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736965    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736968    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736970    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736973    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736976    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736978    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736981    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736986    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736989    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736992    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736995    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.736998    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737000    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737003    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737006    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.740283 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737008    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737372    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737379    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737383    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737386    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737389    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737392    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737395    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737399    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737403    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737406    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737409    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737412    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737414    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737417    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737420    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737422    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737425    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737428    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.740774 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737430    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737433    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737436    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737438    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737440    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737443    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737446    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737449    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737451    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737454    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737456    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737459    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737462    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737464    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737467    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737469    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737472    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737474    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737477    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737479    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.741250 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737482    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737484    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737487    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737490    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737494    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737496    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737499    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737502    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737504    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737507    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737509    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737512    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737514    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737517    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737520    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737522    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737525    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737528    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737530    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737533    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.741717 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737535    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737538    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737541    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737543    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737546    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737548    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737551    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737553    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737556    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737558    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737561    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737563    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737565    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737568    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737571    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737573    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737576    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737579    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737581    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737584    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.742224 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737586    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737589    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737592    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737595    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737599    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737602    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737604    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737607    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737609    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737618    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737620    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737622    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737625    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737628    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737630    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737632    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737635    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737637    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737640    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.742712 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737643    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.737645    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737716    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737722    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737731    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737738    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737744    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737748    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737753    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737760    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737766    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737771    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737775    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737780    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737784    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737789    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737793    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737797    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737802    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737806    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737812    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737816    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737822    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737826    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:02:11.743236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737831    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737837    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737842    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737847    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737852    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737856    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737861    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737865    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737870    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737875    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737882    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737886    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737890    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737894    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737898    2577 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737902    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737909    2577 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737914    2577 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737919    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737924    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737929    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737934    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737939    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737945    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737949    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:02:11.743834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737955    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737961    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737965    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737971    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737975    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737981    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737987    2577 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737993    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.737999    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738004    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738009    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738014    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738020    2577 flags.go:64] FLAG: --help="false"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738025    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-99.ec2.internal"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738033    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738039    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738044    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738049    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738055    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738060    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738064    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738068    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738073    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:02:11.744870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738078    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738083    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738088    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738092    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738096    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738099    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738102    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738105    2577 flags.go:64] FLAG: --lock-file=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738108    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738110    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738113    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738131    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738134    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738137    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738140    2577 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738143    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738147    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738149    2577 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738152    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738157    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738160    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738164    2577 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738167    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738171    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738174    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:02:11.745928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738177    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738180    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738183    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738186    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738193    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738196    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738200    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738203    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738206    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738212    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738215    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738218    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738221    2577 flags.go:64] FLAG: --port="10250"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738225    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738228    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a7c56e8af96d20f8"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738231    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738234    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738237    2577 flags.go:64] FLAG: --register-node="true"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738240    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738249    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738252    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738255    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738257    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738261    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738264    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:02:11.746624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738267    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738270    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738273    2577 flags.go:64] FLAG: --runonce="false"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738275    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738278    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738281    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738286    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738289    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738292    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738295    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738298    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738301    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738304    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738307    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738309    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738314    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738317    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738320    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738326    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738329    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738332    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738336    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738339    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738342    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738345    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:02:11.747523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738348    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738351    2577 flags.go:64] FLAG: --v="2"
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738355    2577 flags.go:64] FLAG: --version="false"
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738358    2577 flags.go:64] FLAG: --vmodule=""
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738362    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738365    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738491    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738495    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738498    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738501    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738503    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738506    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738509    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738512    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738515    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738518    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738521    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738523    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738526    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738528    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738531    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.748527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738535    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738540    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738543    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738546    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738548    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738551    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738553    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738557    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738559    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738562    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738565    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738567    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738570    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738572    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738575    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738577    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738580    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738583    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738585    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738589    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.749248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738592    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738595    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738598    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738601    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738604    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738607    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738610    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738612    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738615    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738617    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738620    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738622    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738625    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738629    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738631    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738634    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738637    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738639    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738642    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738644    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.750069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738648    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738651    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738653    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738655    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738658    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738661    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738663    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738666    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738669    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738671    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738674    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738676    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738679    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738681    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738684    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738686    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738690    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738693    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738696    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738698    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.750715 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738701    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738703    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738706    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738708    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738711    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738714    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738717    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738720    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738723    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738725    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738728    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738730    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738732    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738736    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738738    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738741    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738743    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738746    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738748    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.751352 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738751    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738754    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738758    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738760    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.738763    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.738772    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.746032    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.746051    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.746899    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.746913    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747000    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747008    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747014    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747020    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.751847 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747024    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747028    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747033    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747037    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747041    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747045    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747050    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747054    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747058    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747063    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747068    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747072    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747076    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747080    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747084    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747089    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747092    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747097    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747101    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747106    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.752284 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747110    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747114    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747133    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747138    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747142    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747148    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747152    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747156    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747160    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747165    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747170    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747174    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747178    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747185    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747191    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747195    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747200    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747204    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747208    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747213    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.752767 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747217    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747221    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747225    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747229    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747233    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747238    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747242    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747246    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747250    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747254    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747259    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747262    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747268    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747272    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747276    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747280    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747285    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747289    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747294    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747298    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.753285 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747303    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747307    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747311    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747315    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747319    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747323    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747327    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747331    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747335    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747340    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747344    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747348    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747352    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747356    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747360    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747364    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747368    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747375    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747380    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.753766 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747385    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747389    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747394    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747398    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747405    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747410    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747415    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747419    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747424    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747428    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747433    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747437    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747441    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747446    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747450    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.754236 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747454    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.747462    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747638    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747646    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747651    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747656    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747660    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747665    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747669    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747674    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747678    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747682    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747686    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747691    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:11.754619 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747696    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747700    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747705    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747709    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747713    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747717    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747721    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747725    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747729    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747734    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747738    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747742    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747746    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747750    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747754    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747759    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747764    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747768    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747772    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747776    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:11.754956 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747781    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747785    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747789    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747793    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747797    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747801    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747805    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747809    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747814    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747818    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747823    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747827    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747831    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747835    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747839    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747843    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747847    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747851    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747856    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747860    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:11.755455 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747864    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747868    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747872    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747877    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747881    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747885    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747890    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747894    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747898    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747903    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747907    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747912    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747916    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747920    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747924    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747928    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747932    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747936    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747940    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747945    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:11.755959 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747950    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747956    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747960    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747966    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747970    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747974    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747979    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747983    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747988    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747992    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.747996    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748000    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748004    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748009    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748016    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748021    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748026    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748031    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748035    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:11.756538 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748040    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748044    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748048    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748054    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748058    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748062    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748066    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:11.748072    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.748080    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.748260    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.750901    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.752084    2577 server.go:1019] "Starting client certificate rotation"
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.752150    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:11.757286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.752692    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:11.757642 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.754278    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:11.759365 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.759344    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:11.768216 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.768201    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:02:11.771247 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.771232    2577 log.go:25] "Validated CRI v1 image API"
Jul 28 19:02:11.772713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.772701    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:02:11.774659 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.774639    2577 fs.go:135] Filesystem UUIDs: map[42ebb7c0-650e-4398-888d-3744572ae27b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e55861a4-0030-4833-a44a-3cd5e234960e:/dev/nvme0n1p4]
Jul 28 19:02:11.774700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.774659    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:02:11.779186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.779066    2577 manager.go:216] Machine: {Timestamp:2026-07-28 19:02:11.778597982 +0000 UTC m=+0.267613476 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100545 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2847fd1cf4c0fe9ea1c8f53955d478 SystemUUID:ec2847fd-1cf4-c0fe-9ea1-c8f53955d478 BootID:8cadf0a3-1996-41c2-b8c5-f9137ff36523 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:80:b2:7a:47:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:80:b2:7a:47:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:29:9a:70:39:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:02:11.779186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.779181    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:02:11.779289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.779254    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:02:11.781637 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.781608    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:02:11.781764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.781640    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:02:11.782223 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.782214    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:02:11.782256 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.782226    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:02:11.782256 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.782234    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:11.782939 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.782930    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:11.783349 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.783332    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:11.783679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.783667    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:02:11.786392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786381    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:11.786505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786495    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:02:11.786549 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786525    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:02:11.786549 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786535    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:02:11.786549 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786548    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:02:11.786660 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786557    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:02:11.786660 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786565    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:02:11.786889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786878    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:11.786918 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.786897    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:11.788963 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.788941    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:02:11.790931 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.790913    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:02:11.790992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.790956    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:02:11.792325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792311    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792331    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792340    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792352    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792364    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792372    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:02:11.792384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792383    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:02:11.792574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792391    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:02:11.792574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792411    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:02:11.792574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792418    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:02:11.792574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792430    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:02:11.792574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.792438    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:02:11.793165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.793152    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:02:11.793210 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.793167    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:02:11.796891 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.796875    2577 server.go:1268] "Started kubelet"
Jul 28 19:02:11.797112 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.796922    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:02:11.797306 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.797094    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:02:11.797482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.797463    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:02:11.797840 ip-10-0-142-99 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:02:11.800456 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.800381    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:02:11.800639 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.800614    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:02:11.800969 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.800943    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:02:11.801280 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.801235    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:02:11.802335 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.802313    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:02:11.802616 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.802577    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:02:11.806771 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.806002    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-99.ec2.internal.18c6895c4b917ea9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-99.ec2.internal,UID:ip-10-0-142-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-99.ec2.internal,},FirstTimestamp:2026-07-28 19:02:11.796803241 +0000 UTC m=+0.285818737,LastTimestamp:2026-07-28 19:02:11.796803241 +0000 UTC m=+0.285818737,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-99.ec2.internal,}"
Jul 28 19:02:11.807772 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.807756    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:02:11.807841 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.807801    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:11.807944 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.807917    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:11.808520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.808496    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:02:11.808520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.808516    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:02:11.808742 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.808591    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:02:11.808817 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.808745    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:02:11.808953 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.808936    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:02:11.809144 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809111    2577 factory.go:55] Registering systemd factory
Jul 28 19:02:11.809204 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809146    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:02:11.809416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809404    2577 factory.go:153] Registering CRI-O factory
Jul 28 19:02:11.809416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809417    2577 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:02:11.809515 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809457    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:02:11.809515 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809475    2577 factory.go:103] Registering Raw factory
Jul 28 19:02:11.809515 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809487    2577 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:02:11.809824 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.809806    2577 manager.go:349] Starting recovery of all containers
Jul 28 19:02:11.814429 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.814403    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:02:11.814543 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.814518    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:02:11.820357 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.820341    2577 manager.go:354] Recovery completed
Jul 28 19:02:11.824448 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.824435    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:11.826027 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826013    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:11.826081 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826040    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:11.826081 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826050    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:11.826583 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826568    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:02:11.826583 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826581    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:02:11.826711 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826574    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:11.826711 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.826596    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:11.827679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.827663    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:11.827740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.827690    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:11.827740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.827703    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:11.828962 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.828874    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-99.ec2.internal.18c6895c4d4f6c4a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-99.ec2.internal,UID:ip-10-0-142-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-99.ec2.internal,},FirstTimestamp:2026-07-28 19:02:11.826027594 +0000 UTC m=+0.315043088,LastTimestamp:2026-07-28 19:02:11.826027594 +0000 UTC m=+0.315043088,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-99.ec2.internal,}"
Jul 28 19:02:11.829150 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.829131    2577 policy_none.go:49] "None policy: Start"
Jul 28 19:02:11.829150 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.829153    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:02:11.829279 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.829167    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:02:11.831158 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.831148    2577 policy_none.go:47] "Start"
Jul 28 19:02:11.834811 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.834694    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hkzb7"
Jul 28 19:02:11.837476 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.837404    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-99.ec2.internal.18c6895c4d4fb09d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-99.ec2.internal,UID:ip-10-0-142-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-99.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-99.ec2.internal,},FirstTimestamp:2026-07-28 19:02:11.826045085 +0000 UTC m=+0.315060578,LastTimestamp:2026-07-28 19:02:11.826045085 +0000 UTC m=+0.315060578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-99.ec2.internal,}"
Jul 28 19:02:11.844675 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.842948    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hkzb7"
Jul 28 19:02:11.847579 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.847512    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-99.ec2.internal.18c6895c4d4fd548  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-99.ec2.internal,UID:ip-10-0-142-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-99.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-99.ec2.internal,},FirstTimestamp:2026-07-28 19:02:11.826054472 +0000 UTC m=+0.315069966,LastTimestamp:2026-07-28 19:02:11.826054472 +0000 UTC m=+0.315069966,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-99.ec2.internal,}"
Jul 28 19:02:11.854702 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.854688    2577 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:02:11.854764 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.854715    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:02:11.854764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.854730    2577 server.go:87] "Starting device plugin registration server"
Jul 28 19:02:11.854974 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.854963    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:02:11.855016 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.854976    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:02:11.855100 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.855084    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:02:11.855188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.855171    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:02:11.855188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.855187    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:02:11.855756 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.855738    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:02:11.855837 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.855780    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:11.955356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.955327    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:11.957445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.956293    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:11.957445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.956322    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:11.957445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.956333    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:11.957445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.956358    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-99.ec2.internal"
Jul 28 19:02:11.966417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.966398    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-99.ec2.internal"
Jul 28 19:02:11.966460 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.966422    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-99.ec2.internal\": node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:11.997209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.997110    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:02:11.998274 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.998257    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:02:11.998360 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.998296    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:02:11.998360 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:11.998321    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:02:11.998451 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:11.998362    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:02:12.000488 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.000469    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:12.004287 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.004268    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:12.098658 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.098621    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"]
Jul 28 19:02:12.098796 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.098697    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:12.100289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.100269    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:12.100375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.100300    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:12.100375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.100309    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:12.100565 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.100550    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:12.102545 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.102533    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:12.102645 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.102630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.102690 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.102656    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:12.103275 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103260    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:12.103330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103262    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:12.103330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103313    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:12.103330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103325    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:12.103417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103289    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:12.103417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.103359    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:12.105537 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.105522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.105584 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.105553    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:12.106663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.106648    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:12.106712 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.106678    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:12.106712 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.106692    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:12.109714 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.109695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bb065f00ffb5abb38193a79ef9f99064-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"bb065f00ffb5abb38193a79ef9f99064\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.109794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.109728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.109794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.109752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.120653 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.120631    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-99.ec2.internal\" not found" node="ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.124458 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.124442    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-99.ec2.internal\" not found" node="ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.200789 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.200760    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:12.210072 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bb065f00ffb5abb38193a79ef9f99064-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"bb065f00ffb5abb38193a79ef9f99064\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.210182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.210182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.210262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bb065f00ffb5abb38193a79ef9f99064-config\") pod \"kube-apiserver-proxy-ip-10-0-142-99.ec2.internal\" (UID: \"bb065f00ffb5abb38193a79ef9f99064\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.210262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.210262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.210236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48cf1ad2ed4c6ccccfee7c9798cc69e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal\" (UID: \"48cf1ad2ed4c6ccccfee7c9798cc69e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:12.301248 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.301168    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:12.401688 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:12.401646    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:12.423182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:12.423160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:13.475155 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.475136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:13.549146 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:13.549110    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:13.649678 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:13.649597    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:13.750174 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:13.750142    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:13.798872 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.798842    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:02:13.799409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.798976    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:02:13.841651 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.841620    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:13.850677 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:13.850658    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:13.901767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.901684    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 18:57:12 +0000 UTC" deadline="2028-01-25 21:14:20.243958325 +0000 UTC"
Jul 28 19:02:13.901767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.901717    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13106h12m6.342245561s"
Jul 28 19:02:13.901767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.901742    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:13.930501 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.930474    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:13.931691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.931672    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:13.931801 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.931701    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:13.931801 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.931711    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:13.951258 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:13.951229    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:13.983673 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:13.983644    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cf1ad2ed4c6ccccfee7c9798cc69e6.slice/crio-952fc28cef13a300b9a24a46503cf6b915cd80ebd1e234f0a2433cd502391064 WatchSource:0}: Error finding container 952fc28cef13a300b9a24a46503cf6b915cd80ebd1e234f0a2433cd502391064: Status 404 returned error can't find the container with id 952fc28cef13a300b9a24a46503cf6b915cd80ebd1e234f0a2433cd502391064
Jul 28 19:02:13.988344 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:13.988329    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:02:14.048986 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.048941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"48cf1ad2ed4c6ccccfee7c9798cc69e6","Type":"ContainerStarted","Data":"952fc28cef13a300b9a24a46503cf6b915cd80ebd1e234f0a2433cd502391064"}
Jul 28 19:02:14.052120 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.052095    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:14.152657 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.152565    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:14.235963 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:14.235936    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb065f00ffb5abb38193a79ef9f99064.slice/crio-54a793d683ee5d8e6182580063ab325f88fa11ca1f4ed179ce9c9304ec7c9b68 WatchSource:0}: Error finding container 54a793d683ee5d8e6182580063ab325f88fa11ca1f4ed179ce9c9304ec7c9b68: Status 404 returned error can't find the container with id 54a793d683ee5d8e6182580063ab325f88fa11ca1f4ed179ce9c9304ec7c9b68
Jul 28 19:02:14.253579 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.253559    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-99.ec2.internal\" not found"
Jul 28 19:02:14.289583 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.289559    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:14.355708 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.355685    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:14.366372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.366353    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:14.367138 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.367124    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal"
Jul 28 19:02:14.379082 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.379065    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:14.834795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.834766    2577 apiserver.go:52] "Watching apiserver"
Jul 28 19:02:14.844425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.844398    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:02:14.845341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.845317    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-2gt4r","openshift-image-registry/node-ca-8m52c","openshift-multus/multus-additional-cni-plugins-tq6kq","openshift-network-diagnostics/network-check-target-xpd7r","openshift-network-operator/iptables-alerter-r7hnn","openshift-ovn-kubernetes/ovnkube-node-hd2xp","kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k","openshift-cluster-node-tuning-operator/tuned-wjlz5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal","openshift-multus/multus-kmc28","openshift-multus/network-metrics-daemon-5g9jv","kube-system/global-pull-secret-syncer-k5lrq","kube-system/konnectivity-agent-msf7f"]
Jul 28 19:02:14.847952 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.847927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.848040 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.848021    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:14.850019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.850001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.852269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.852245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.852399 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.852377    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.852399 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.852395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.852608 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.852413    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nrqz4\""
Jul 28 19:02:14.852608 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.852492    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:02:14.854392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:14.854484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854429    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.854484 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.854443    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:14.854935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854778    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.854935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854813    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p7xcd\""
Jul 28 19:02:14.854935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854839    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:02:14.854935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854851    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:02:14.854935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.854779    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:02:14.855911 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.855886    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:14.856799 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.856773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.858901 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.858882    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.858989 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.858967    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.859050 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.859034    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:02:14.859166 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.859152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.859298 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.859282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fkv95\""
Jul 28 19:02:14.861156 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:02:14.861904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861492    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.861904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861634    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:02:14.861904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861733    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:02:14.861904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.861904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.861905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:14.862175 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.862099    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:02:14.862175 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.862159    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q2rxx\""
Jul 28 19:02:14.864070 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.863958    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.864954 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.864938    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.865140 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865121    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glwzt\""
Jul 28 19:02:14.865234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865183    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.865795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cnibin\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.865883 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865780    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.865883 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-os-release\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.865883 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.865883 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f801956-5ca1-44d3-8dd4-3555edde9cb4-iptables-alerter-script\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-kubelet-config\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-dbus\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5sgw\" (UniqueName: \"kubernetes.io/projected/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-kube-api-access-m5sgw\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.865985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f801956-5ca1-44d3-8dd4-3555edde9cb4-host-slash\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.866066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bdaf7e7e-023f-494c-9126-284b229cdf69-serviceca\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sbfp\" (UniqueName: \"kubernetes.io/projected/bdaf7e7e-023f-494c-9126-284b229cdf69-kube-api-access-8sbfp\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-system-cni-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4nm\" (UniqueName: \"kubernetes.io/projected/8f801956-5ca1-44d3-8dd4-3555edde9cb4-kube-api-access-vj4nm\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdaf7e7e-023f-494c-9126-284b229cdf69-host\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.866492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.866769 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.866769 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jkfjf\""
Jul 28 19:02:14.866971 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.867037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.866961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:02:14.868767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.868749    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:14.868850 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.868773    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cpzxr\""
Jul 28 19:02:14.868850 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.868760    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:14.869099 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.869086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:14.869471 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.869454    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:14.871508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.871147    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6sdsm\""
Jul 28 19:02:14.871508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.871445    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:02:14.871763 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.871747    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:02:14.874493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.874472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.876948 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.876931    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vl5q9\""
Jul 28 19:02:14.877037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.876963    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:02:14.877478 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.877459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:14.877563 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.877527    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:14.890053 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.890032    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tp2fx"
Jul 28 19:02:14.897511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.897489    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tp2fx"
Jul 28 19:02:14.956871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.956847    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:02:14.966642 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-os-release\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.966642 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-kubelet-config\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-kubelet-config\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-os-release\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-etc-selinux\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-sys-fs\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.966845 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-kubernetes\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-system-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f801956-5ca1-44d3-8dd4-3555edde9cb4-host-slash\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bdaf7e7e-023f-494c-9126-284b229cdf69-serviceca\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sbfp\" (UniqueName: \"kubernetes.io/projected/bdaf7e7e-023f-494c-9126-284b229cdf69-kube-api-access-8sbfp\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-sys\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.966987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-tmp\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-netd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-cni-binary-copy\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-systemd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-bin\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-ovn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-config\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-os-release\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-kubelet\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-etc-kubernetes\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-dbus\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f801956-5ca1-44d3-8dd4-3555edde9cb4-host-slash\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysconfig\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-systemd\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmv9w\" (UniqueName: \"kubernetes.io/projected/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-kube-api-access-nmv9w\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-kubelet\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-node-log\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:14.967654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfjdb\" (UniqueName: \"kubernetes.io/projected/b491cd82-fcb5-4b98-9110-70b75b8edbce-kube-api-access-zfjdb\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-log-socket\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-conf-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-dbus\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4nm\" (UniqueName: \"kubernetes.io/projected/8f801956-5ca1-44d3-8dd4-3555edde9cb4-kube-api-access-vj4nm\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlf68\" (UniqueName: \"kubernetes.io/projected/ea2fbe2e-e91e-416c-89d5-2f621a27990b-kube-api-access-wlf68\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.967799    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-k8s-cni-cncf-io\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bdaf7e7e-023f-494c-9126-284b229cdf69-serviceca\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jdb2\" (UniqueName: \"kubernetes.io/projected/e053161e-67d0-4a87-8490-b3b54a7fe458-kube-api-access-9jdb2\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.967857    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:15.467838236 +0000 UTC m=+2.909758723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.967993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-var-lib-kubelet\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d16f0ab-d2a4-461b-9291-d24b001c6549-konnectivity-ca\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4eac62bc-bac0-4579-b519-8e202835eadf-ovn-node-metrics-cert\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-device-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.968433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-lib-modules\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdkzn\" (UniqueName: \"kubernetes.io/projected/4eac62bc-bac0-4579-b519-8e202835eadf-kube-api-access-mdkzn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-socket-dir-parent\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-daemon-config\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-host\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-systemd-units\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968598    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-netns\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-bin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdaf7e7e-023f-494c-9126-284b229cdf69-host\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d16f0ab-d2a4-461b-9291-d24b001c6549-agent-certs\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdaf7e7e-023f-494c-9126-284b229cdf69-host\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-etc-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-netns\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a1b57d-0dca-4762-97af-5cff080bacde-tmp-dir\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:14.969030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-registration-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-modprobe-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.968907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f801956-5ca1-44d3-8dd4-3555edde9cb4-iptables-alerter-script\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxwt\" (UniqueName: \"kubernetes.io/projected/34a1b57d-0dca-4762-97af-5cff080bacde-kube-api-access-twxwt\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sgw\" (UniqueName: \"kubernetes.io/projected/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-kube-api-access-m5sgw\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-conf\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-multus\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34a1b57d-0dca-4762-97af-5cff080bacde-hosts-file\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-system-cni-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-tuned\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-slash\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.969716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-system-cni-dir\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-var-lib-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-script-lib\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-cnibin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f801956-5ca1-44d3-8dd4-3555edde9cb4-iptables-alerter-script\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-hostroot\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-multus-certs\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-socket-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-run\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-env-overrides\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cnibin\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.970196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.969739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-cnibin\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.973670 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.973646    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:02:14.974080 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.974063    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:14.974181 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.974084    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:14.974181 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.974097    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:14.974181 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:14.974168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:15.474135687 +0000 UTC m=+2.916056190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:14.977380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.977352    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5sgw\" (UniqueName: \"kubernetes.io/projected/40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7-kube-api-access-m5sgw\") pod \"multus-additional-cni-plugins-tq6kq\" (UID: \"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7\") " pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:14.977380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.977364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sbfp\" (UniqueName: \"kubernetes.io/projected/bdaf7e7e-023f-494c-9126-284b229cdf69-kube-api-access-8sbfp\") pod \"node-ca-8m52c\" (UID: \"bdaf7e7e-023f-494c-9126-284b229cdf69\") " pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:14.977503 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:14.977364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4nm\" (UniqueName: \"kubernetes.io/projected/8f801956-5ca1-44d3-8dd4-3555edde9cb4-kube-api-access-vj4nm\") pod \"iptables-alerter-r7hnn\" (UID: \"8f801956-5ca1-44d3-8dd4-3555edde9cb4\") " pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:15.051767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.051722    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" event={"ID":"bb065f00ffb5abb38193a79ef9f99064","Type":"ContainerStarted","Data":"54a793d683ee5d8e6182580063ab325f88fa11ca1f4ed179ce9c9304ec7c9b68"}
Jul 28 19:02:15.070104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-multus\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34a1b57d-0dca-4762-97af-5cff080bacde-hosts-file\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-tuned\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-slash\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-var-lib-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-slash\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070293 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-script-lib\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-cnibin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-hostroot\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-multus-certs\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-socket-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-run\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-env-overrides\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-etc-selinux\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-sys-fs\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-kubernetes\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-multus\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-system-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-sys\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.070644 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-system-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-tmp\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-netd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-cnibin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-cni-binary-copy\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-systemd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-hostroot\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-bin\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-multus-certs\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-ovn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-script-lib\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-config\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-socket-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34a1b57d-0dca-4762-97af-5cff080bacde-hosts-file\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-os-release\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-run\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-kubelet\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.071405 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.070997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-etc-kubernetes\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysconfig\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.071039    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-systemd\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmv9w\" (UniqueName: \"kubernetes.io/projected/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-kube-api-access-nmv9w\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.071101    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:15.571086391 +0000 UTC m=+3.013006894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-kubelet\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-node-log\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfjdb\" (UniqueName: \"kubernetes.io/projected/b491cd82-fcb5-4b98-9110-70b75b8edbce-kube-api-access-zfjdb\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-log-socket\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-conf-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf68\" (UniqueName: \"kubernetes.io/projected/ea2fbe2e-e91e-416c-89d5-2f621a27990b-kube-api-access-wlf68\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-k8s-cni-cncf-io\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jdb2\" (UniqueName: \"kubernetes.io/projected/e053161e-67d0-4a87-8490-b3b54a7fe458-kube-api-access-9jdb2\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-ovnkube-config\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-var-lib-kubelet\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-os-release\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.072235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-kubelet\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d16f0ab-d2a4-461b-9291-d24b001c6549-konnectivity-ca\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4eac62bc-bac0-4579-b519-8e202835eadf-ovn-node-metrics-cert\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-device-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-lib-modules\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-device-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdkzn\" (UniqueName: \"kubernetes.io/projected/4eac62bc-bac0-4579-b519-8e202835eadf-kube-api-access-mdkzn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-etc-kubernetes\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-socket-dir-parent\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysconfig\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-daemon-config\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-systemd\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-host\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-systemd-units\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-sys\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-systemd-units\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-netns\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-bin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d16f0ab-d2a4-461b-9291-d24b001c6549-agent-certs\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-etc-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-netns\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a1b57d-0dca-4762-97af-5cff080bacde-tmp-dir\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-registration-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.071971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-modprobe-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxwt\" (UniqueName: \"kubernetes.io/projected/34a1b57d-0dca-4762-97af-5cff080bacde-kube-api-access-twxwt\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-node-log\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-var-lib-cni-bin\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-lib-modules\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-k8s-cni-cncf-io\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-conf\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-etc-selinux\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-kubernetes\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.073774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-cni-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-conf\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-log-socket\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4eac62bc-bac0-4579-b519-8e202835eadf-env-overrides\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-conf-dir\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-socket-dir-parent\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-sys-fs\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-var-lib-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072639    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-var-lib-kubelet\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-kubelet\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b491cd82-fcb5-4b98-9110-70b75b8edbce-registration-dir\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-host\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-etc-openvswitch\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e053161e-67d0-4a87-8490-b3b54a7fe458-host-run-netns\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-modprobe-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-multus-daemon-config\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-netns\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.074619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-bin\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-sysctl-d\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-systemd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-run-ovn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.072984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4eac62bc-bac0-4579-b519-8e202835eadf-host-cni-netd\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.073314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e053161e-67d0-4a87-8490-b3b54a7fe458-cni-binary-copy\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.073471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d16f0ab-d2a4-461b-9291-d24b001c6549-konnectivity-ca\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.073755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-etc-tuned\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.075329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.074493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a1b57d-0dca-4762-97af-5cff080bacde-tmp-dir\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.075948 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.075927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-tmp\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.076060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.075999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4eac62bc-bac0-4579-b519-8e202835eadf-ovn-node-metrics-cert\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.076124 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.076108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d16f0ab-d2a4-461b-9291-d24b001c6549-agent-certs\") pod \"konnectivity-agent-msf7f\" (UID: \"7d16f0ab-d2a4-461b-9291-d24b001c6549\") " pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:15.078678 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.078653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmv9w\" (UniqueName: \"kubernetes.io/projected/f8d95d63-211e-4831-b2bc-ed2cb121c1f9-kube-api-access-nmv9w\") pod \"tuned-wjlz5\" (UID: \"f8d95d63-211e-4831-b2bc-ed2cb121c1f9\") " pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.082452 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.082431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfjdb\" (UniqueName: \"kubernetes.io/projected/b491cd82-fcb5-4b98-9110-70b75b8edbce-kube-api-access-zfjdb\") pod \"aws-ebs-csi-driver-node-lwg5k\" (UID: \"b491cd82-fcb5-4b98-9110-70b75b8edbce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.082711 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.082691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdkzn\" (UniqueName: \"kubernetes.io/projected/4eac62bc-bac0-4579-b519-8e202835eadf-kube-api-access-mdkzn\") pod \"ovnkube-node-hd2xp\" (UID: \"4eac62bc-bac0-4579-b519-8e202835eadf\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.083039 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.083020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlf68\" (UniqueName: \"kubernetes.io/projected/ea2fbe2e-e91e-416c-89d5-2f621a27990b-kube-api-access-wlf68\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:15.083495 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.083475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxwt\" (UniqueName: \"kubernetes.io/projected/34a1b57d-0dca-4762-97af-5cff080bacde-kube-api-access-twxwt\") pod \"node-resolver-2gt4r\" (UID: \"34a1b57d-0dca-4762-97af-5cff080bacde\") " pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.083578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.083523    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jdb2\" (UniqueName: \"kubernetes.io/projected/e053161e-67d0-4a87-8490-b3b54a7fe458-kube-api-access-9jdb2\") pod \"multus-kmc28\" (UID: \"e053161e-67d0-4a87-8490-b3b54a7fe458\") " pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.152599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.152528    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:15.160468 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.160445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8m52c"
Jul 28 19:02:15.168302 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.168280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tq6kq"
Jul 28 19:02:15.169046 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.169023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdaf7e7e_023f_494c_9126_284b229cdf69.slice/crio-e86fbbd8be7bf63930b27e12a958f31ce18697b65aa90047be6f0c934050e9e0 WatchSource:0}: Error finding container e86fbbd8be7bf63930b27e12a958f31ce18697b65aa90047be6f0c934050e9e0: Status 404 returned error can't find the container with id e86fbbd8be7bf63930b27e12a958f31ce18697b65aa90047be6f0c934050e9e0
Jul 28 19:02:15.177317 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.177294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7hnn"
Jul 28 19:02:15.177868 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.177850    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a25323_a3d4_4cfc_9702_d4c3d1bf5ce7.slice/crio-0292f889a2fbc6dc0c43a705b9020fb7053ec399e65a4b9afb629e4a75417c80 WatchSource:0}: Error finding container 0292f889a2fbc6dc0c43a705b9020fb7053ec399e65a4b9afb629e4a75417c80: Status 404 returned error can't find the container with id 0292f889a2fbc6dc0c43a705b9020fb7053ec399e65a4b9afb629e4a75417c80
Jul 28 19:02:15.184134 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.184116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:15.187649 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.187622    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f801956_5ca1_44d3_8dd4_3555edde9cb4.slice/crio-384f7c2a4a57260743c378270548e15595dfb345a39e982f03bfcd980a738361 WatchSource:0}: Error finding container 384f7c2a4a57260743c378270548e15595dfb345a39e982f03bfcd980a738361: Status 404 returned error can't find the container with id 384f7c2a4a57260743c378270548e15595dfb345a39e982f03bfcd980a738361
Jul 28 19:02:15.190141 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.190119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2gt4r"
Jul 28 19:02:15.192818 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.192798    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eac62bc_bac0_4579_b519_8e202835eadf.slice/crio-b4ab1e8b124c94642a0dd178046f7ef1c30c0fd4ec0c3d5f6a44c7686774b680 WatchSource:0}: Error finding container b4ab1e8b124c94642a0dd178046f7ef1c30c0fd4ec0c3d5f6a44c7686774b680: Status 404 returned error can't find the container with id b4ab1e8b124c94642a0dd178046f7ef1c30c0fd4ec0c3d5f6a44c7686774b680
Jul 28 19:02:15.196413 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.196391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k"
Jul 28 19:02:15.201436 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.201413    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a1b57d_0dca_4762_97af_5cff080bacde.slice/crio-a28559d3d0b60c2c30c5316c4e71587b4e2af04c2c1f28bc063e3b666f52155d WatchSource:0}: Error finding container a28559d3d0b60c2c30c5316c4e71587b4e2af04c2c1f28bc063e3b666f52155d: Status 404 returned error can't find the container with id a28559d3d0b60c2c30c5316c4e71587b4e2af04c2c1f28bc063e3b666f52155d
Jul 28 19:02:15.202942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.202926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wjlz5"
Jul 28 19:02:15.212067 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.212045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:15.212327 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.212306    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d95d63_211e_4831_b2bc_ed2cb121c1f9.slice/crio-1404b82be085eb7851e62f714dd3703c52991c5dcfd3be00a035d27b238d6508 WatchSource:0}: Error finding container 1404b82be085eb7851e62f714dd3703c52991c5dcfd3be00a035d27b238d6508: Status 404 returned error can't find the container with id 1404b82be085eb7851e62f714dd3703c52991c5dcfd3be00a035d27b238d6508
Jul 28 19:02:15.216791 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.216774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kmc28"
Jul 28 19:02:15.221333 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.221312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d16f0ab_d2a4_461b_9291_d24b001c6549.slice/crio-a0f0e1361bc78d650300bd71c47f31e2be1df7d95d0c92387ae93098a24c5d26 WatchSource:0}: Error finding container a0f0e1361bc78d650300bd71c47f31e2be1df7d95d0c92387ae93098a24c5d26: Status 404 returned error can't find the container with id a0f0e1361bc78d650300bd71c47f31e2be1df7d95d0c92387ae93098a24c5d26
Jul 28 19:02:15.226286 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:15.226265    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode053161e_67d0_4a87_8490_b3b54a7fe458.slice/crio-2c46e67a29504418eb64ea7d3a4472d6addc15884663892da72a8d11637908a3 WatchSource:0}: Error finding container 2c46e67a29504418eb64ea7d3a4472d6addc15884663892da72a8d11637908a3: Status 404 returned error can't find the container with id 2c46e67a29504418eb64ea7d3a4472d6addc15884663892da72a8d11637908a3
Jul 28 19:02:15.474494 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.474315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:15.474617 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.474505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:15.474617 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474459    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:15.474713 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474617    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:15.474713 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474629    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:15.474713 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474637    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:15.474713 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474670    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:16.474632622 +0000 UTC m=+3.916553114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:15.474713 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.474692    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:16.474679392 +0000 UTC m=+3.916599877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:15.575034 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.575004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:15.575265 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.575227    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:15.575349 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:15.575288    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:16.575269637 +0000 UTC m=+4.017190137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:15.900882 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.900657    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:14 +0000 UTC" deadline="2028-01-14 02:50:46.757754715 +0000 UTC"
Jul 28 19:02:15.900882 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:15.900718    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12823h48m30.857040507s"
Jul 28 19:02:16.065680 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.065641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kmc28" event={"ID":"e053161e-67d0-4a87-8490-b3b54a7fe458","Type":"ContainerStarted","Data":"2c46e67a29504418eb64ea7d3a4472d6addc15884663892da72a8d11637908a3"}
Jul 28 19:02:16.071026 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.070911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msf7f" event={"ID":"7d16f0ab-d2a4-461b-9291-d24b001c6549","Type":"ContainerStarted","Data":"a0f0e1361bc78d650300bd71c47f31e2be1df7d95d0c92387ae93098a24c5d26"}
Jul 28 19:02:16.082071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.082037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wjlz5" event={"ID":"f8d95d63-211e-4831-b2bc-ed2cb121c1f9","Type":"ContainerStarted","Data":"1404b82be085eb7851e62f714dd3703c52991c5dcfd3be00a035d27b238d6508"}
Jul 28 19:02:16.091061 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.090995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"b4ab1e8b124c94642a0dd178046f7ef1c30c0fd4ec0c3d5f6a44c7686774b680"}
Jul 28 19:02:16.104361 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.104329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7hnn" event={"ID":"8f801956-5ca1-44d3-8dd4-3555edde9cb4","Type":"ContainerStarted","Data":"384f7c2a4a57260743c378270548e15595dfb345a39e982f03bfcd980a738361"}
Jul 28 19:02:16.107318 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.107293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerStarted","Data":"0292f889a2fbc6dc0c43a705b9020fb7053ec399e65a4b9afb629e4a75417c80"}
Jul 28 19:02:16.120879 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.120817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8m52c" event={"ID":"bdaf7e7e-023f-494c-9126-284b229cdf69","Type":"ContainerStarted","Data":"e86fbbd8be7bf63930b27e12a958f31ce18697b65aa90047be6f0c934050e9e0"}
Jul 28 19:02:16.129231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.128305    2577 generic.go:356] "Generic (PLEG): container finished" podID="48cf1ad2ed4c6ccccfee7c9798cc69e6" containerID="876fc5c5008d3dca45c2ece1d9daae4549c0e578ef307c42573dd50fa6bba539" exitCode=0
Jul 28 19:02:16.129231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.128378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"48cf1ad2ed4c6ccccfee7c9798cc69e6","Type":"ContainerDied","Data":"876fc5c5008d3dca45c2ece1d9daae4549c0e578ef307c42573dd50fa6bba539"}
Jul 28 19:02:16.137128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.137097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k" event={"ID":"b491cd82-fcb5-4b98-9110-70b75b8edbce","Type":"ContainerStarted","Data":"7d52ce86fd53083ebbd275095c738f3f8fa3774e19c5ed8bd82cb95e2889b93b"}
Jul 28 19:02:16.146470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.146444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2gt4r" event={"ID":"34a1b57d-0dca-4762-97af-5cff080bacde","Type":"ContainerStarted","Data":"a28559d3d0b60c2c30c5316c4e71587b4e2af04c2c1f28bc063e3b666f52155d"}
Jul 28 19:02:16.491667 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.491562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:16.491667 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.491618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:16.491939 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.491783    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:16.491939 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.491807    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:16.491939 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.491821    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:16.491939 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.491880    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:18.491861151 +0000 UTC m=+5.933781654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:16.492594 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.492299    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:16.492594 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.492351    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:18.492336126 +0000 UTC m=+5.934256618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:16.592776 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.592740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:16.592962 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.592931    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:16.593034 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:16.592997    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:18.592976723 +0000 UTC m=+6.034897213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:16.901387 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.901299    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:14 +0000 UTC" deadline="2028-01-01 22:39:55.705426741 +0000 UTC"
Jul 28 19:02:16.901387 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:16.901341    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12531h37m38.804090716s"
Jul 28 19:02:17.048236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:17.048185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:17.048398 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:17.048345    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:17.048461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:17.048438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:17.048532 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:17.048513    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:17.048592 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:17.048576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:17.048667 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:17.048651    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:17.174313 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:17.173760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" event={"ID":"48cf1ad2ed4c6ccccfee7c9798cc69e6","Type":"ContainerStarted","Data":"4be4688bdf55b0b98a90f66c0f5e6e1e42678b48879651e95925132ae4d37474"}
Jul 28 19:02:18.509289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:18.509249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:18.509308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509464    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509487    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509501    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509561    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:22.509540729 +0000 UTC m=+9.951461232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509633    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:18.509746 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.509665    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:22.509655232 +0000 UTC m=+9.951575731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:18.610822 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:18.610149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:18.610822 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.610364    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:18.610822 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:18.610428    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:22.610410598 +0000 UTC m=+10.052331098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:19.046895 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:19.046197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:19.046895 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:19.046361    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:19.046895 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:19.046743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:19.046895 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:19.046841    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:19.047520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:19.047296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:19.047520 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:19.047386    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:21.045953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:21.045953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:21.046086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:21.046088    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:21.046191    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:21.046461 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:21.046286    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:22.546505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:22.546466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:22.546526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546659    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546675    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546692    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546705    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546742    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:30.546718459 +0000 UTC m=+17.988638948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:22.546978 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.546763    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:30.546753201 +0000 UTC m=+17.988673690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:22.647920 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:22.647876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:22.648092 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.648048    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:22.648147 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:22.648123    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:30.648100623 +0000 UTC m=+18.090021110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:23.046571 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:23.046535    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:23.046571 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:23.046539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:23.047797 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:23.047725    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:23.048179 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:23.048149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:23.048261 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:23.048246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:23.048393 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:23.048373    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:25.046479 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:25.046409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:25.046960 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:25.046546    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:25.046960 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:25.046409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:25.046960 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:25.046658    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:25.046960 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:25.046409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:25.046960 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:25.046722    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:27.045976 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:27.045940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:27.045976 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:27.045978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:27.046534 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:27.045940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:27.046534 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:27.046081    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:27.046534 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:27.046142    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:27.046534 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:27.046240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:29.046786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:29.046978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:29.047686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:29.047884    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:29.047961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:29.048937 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:29.048052    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:30.606447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:30.606415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:30.606459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606571    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606593    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606611    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606624    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606632    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:46.606617665 +0000 UTC m=+34.048538155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:30.606876 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.606673    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:46.606658067 +0000 UTC m=+34.048578573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:30.707515 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:30.707479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:30.707674 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.707657    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:30.707729 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:30.707721    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:02:46.707704144 +0000 UTC m=+34.149624631 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:31.046107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:31.046027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:31.046107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:31.046045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:31.046107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:31.046027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:31.046428 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:31.046172    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:31.046428 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:31.046335    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:31.046428 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:31.046408    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:33.046616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:33.046567    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:33.047067 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:33.046627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:33.047067 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:33.046684    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:33.047067 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:33.046737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:33.047067 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:33.046832    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:33.047067 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:33.046910    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:35.046732 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.046572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:35.047279 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.046577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:35.047279 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:35.046805    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:35.047279 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.046636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:35.047279 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:35.046895    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:35.047279 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:35.046992    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:35.211541 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.211503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kmc28" event={"ID":"e053161e-67d0-4a87-8490-b3b54a7fe458","Type":"ContainerStarted","Data":"6f472758f889cc76478910bb02dd103d70ce466ebdb3a5367329b57624033778"}
Jul 28 19:02:35.213144 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.213122    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msf7f" event={"ID":"7d16f0ab-d2a4-461b-9291-d24b001c6549","Type":"ContainerStarted","Data":"4424ece645c153638fea0fa4da036dd1e1ac9f168175aaa634c8c6924ee318f5"}
Jul 28 19:02:35.214636 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.214612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wjlz5" event={"ID":"f8d95d63-211e-4831-b2bc-ed2cb121c1f9","Type":"ContainerStarted","Data":"90af66d8af51ff26c1e1fb8be8e6d5275eb521ca89e6972b26d15a570cc85ad1"}
Jul 28 19:02:35.217546 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"0568deb31f1b13a449fa0b4cbc8fe28dda937d93abf014f547ac38214fe6433f"}
Jul 28 19:02:35.217640 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"3c138c67090fc5ca8f2d79b58131d69360b1b9faa348650beab2cccf23f75c20"}
Jul 28 19:02:35.217640 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"c35424b3bc1d9586a251ce15d75602025651c3957ab74ccd35dbdd7331e529bf"}
Jul 28 19:02:35.217742 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"8e6540eb45b83552e2526e5b6ff002066b0adb81f3ea11bd91494d1ddddb3c0d"}
Jul 28 19:02:35.217742 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"268ede887423a26789e41554d824c00a3b4a2ed763ccf5f1a7d0d5285b05d862"}
Jul 28 19:02:35.217742 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.217734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"4bbd293702d969dac25a13f2be96623987efb159162db4a80c810a32a378e768"}
Jul 28 19:02:35.219101 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.219081    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="44829fd82625d63160c8bd893e76e3c37d6539f8783e49d959f0347b160eeb93" exitCode=0
Jul 28 19:02:35.219348 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.219145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"44829fd82625d63160c8bd893e76e3c37d6539f8783e49d959f0347b160eeb93"}
Jul 28 19:02:35.220795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.220757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8m52c" event={"ID":"bdaf7e7e-023f-494c-9126-284b229cdf69","Type":"ContainerStarted","Data":"175476a3cbc86822443e96ed25ca3662466a4fdd4961e2033ec5a80c810b6c71"}
Jul 28 19:02:35.222270 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.222247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k" event={"ID":"b491cd82-fcb5-4b98-9110-70b75b8edbce","Type":"ContainerStarted","Data":"adee46254485e76a92148d63fd1120c9a8d0191ac28a9476ab9cbe67ce7344e5"}
Jul 28 19:02:35.223725 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.223703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2gt4r" event={"ID":"34a1b57d-0dca-4762-97af-5cff080bacde","Type":"ContainerStarted","Data":"e28c8cc0a8035ec26e1b64a7e5d1cdafe9f7d289ddcece7a2fb50914b60bea55"}
Jul 28 19:02:35.225207 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.225187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" event={"ID":"bb065f00ffb5abb38193a79ef9f99064","Type":"ContainerStarted","Data":"2ebfc46d548fe601e90eebfddcb23a2499c1dabf479c79672a8bd31b7e72fc93"}
Jul 28 19:02:35.228133 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.228094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kmc28" podStartSLOduration=2.86656361 podStartE2EDuration="22.228068347s" totalImagesPullingTime="19.361504737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.228028554 +0000 UTC m=+22.669949063" watchObservedRunningTime="2026-07-28 19:02:35.228068347 +0000 UTC m=+22.669988848"
Jul 28 19:02:35.228811 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.228773    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-99.ec2.internal" podStartSLOduration=21.228764579 podStartE2EDuration="21.228764579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:17.196899161 +0000 UTC m=+4.638819670" watchObservedRunningTime="2026-07-28 19:02:35.228764579 +0000 UTC m=+22.670685085"
Jul 28 19:02:35.259495 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.259457    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8m52c" podStartSLOduration=2.901470083 podStartE2EDuration="22.259441154s" totalImagesPullingTime="19.357971071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.240880614 +0000 UTC m=+22.682801122" watchObservedRunningTime="2026-07-28 19:02:35.259441154 +0000 UTC m=+22.701361661"
Jul 28 19:02:35.276029 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.275966    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2gt4r" podStartSLOduration=2.94513252 podStartE2EDuration="22.275951597s" totalImagesPullingTime="19.330819077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.275665854 +0000 UTC m=+22.717586361" watchObservedRunningTime="2026-07-28 19:02:35.275951597 +0000 UTC m=+22.717872104"
Jul 28 19:02:35.297963 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.297924    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wjlz5" podStartSLOduration=2.971860358 podStartE2EDuration="22.297912049s" totalImagesPullingTime="19.326051691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.297345844 +0000 UTC m=+22.739266352" watchObservedRunningTime="2026-07-28 19:02:35.297912049 +0000 UTC m=+22.739832556"
Jul 28 19:02:35.314882 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.314844    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-msf7f" podStartSLOduration=10.644792052 podStartE2EDuration="22.314831906s" totalImagesPullingTime="11.670039854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.314528155 +0000 UTC m=+22.756448675" watchObservedRunningTime="2026-07-28 19:02:35.314831906 +0000 UTC m=+22.756752457"
Jul 28 19:02:35.333569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.333529    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-99.ec2.internal" podStartSLOduration=21.33351672 podStartE2EDuration="21.33351672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:35.330387999 +0000 UTC m=+22.772308507" watchObservedRunningTime="2026-07-28 19:02:35.33351672 +0000 UTC m=+22.775437227"
Jul 28 19:02:35.538266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.538182    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:35.539046 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:35.539028    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:36.229159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.229125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7hnn" event={"ID":"8f801956-5ca1-44d3-8dd4-3555edde9cb4","Type":"ContainerStarted","Data":"63d1ae58aae90851518938936eeb4450156b7a61d570352e57665ad95a62cd97"}
Jul 28 19:02:36.244258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.244191    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r7hnn" podStartSLOduration=3.9061623340000002 podStartE2EDuration="23.24417574s" totalImagesPullingTime="19.338013406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:36.244126658 +0000 UTC m=+23.686047167" watchObservedRunningTime="2026-07-28 19:02:36.24417574 +0000 UTC m=+23.686096248"
Jul 28 19:02:36.297589 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.297570    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:02:36.917273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.917141    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:02:36.297585243Z","UUID":"1efa3031-ff5d-490c-923f-17563dc0279d","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:02:36.918754 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.918732    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:02:36.918869 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:36.918772    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:02:37.045720 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:37.045688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:37.045882 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:37.045813    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:37.045882 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:37.045688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:37.046002 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:37.045892    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:37.046002 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:37.045688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:37.046002 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:37.045944    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:37.233559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:37.233487    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:02:37.233559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:37.233488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k" event={"ID":"b491cd82-fcb5-4b98-9110-70b75b8edbce","Type":"ContainerStarted","Data":"5126427b6acf9676d78b59bc773daf6960d832d76b96e2bc82209ea7d8751fc7"}
Jul 28 19:02:38.239800 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:38.239568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"111307bcd8837099f7d911da1336dcc33d2592fd6fc2b7417f087a54938fbfdf"}
Jul 28 19:02:38.241814 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:38.241786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k" event={"ID":"b491cd82-fcb5-4b98-9110-70b75b8edbce","Type":"ContainerStarted","Data":"efa4f2b64d049ca5b50f83e824db372f8bfa758762122c46707acb02007f40be"}
Jul 28 19:02:38.260591 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:38.260552    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lwg5k" podStartSLOduration=3.324504114 podStartE2EDuration="25.260539672s" totalImagesPullingTime="21.936035558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:38.260295181 +0000 UTC m=+25.702215694" watchObservedRunningTime="2026-07-28 19:02:38.260539672 +0000 UTC m=+25.702460179"
Jul 28 19:02:39.046478 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.046444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:39.046665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.046444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:39.046665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.046571    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:39.046665 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:39.046573    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:39.046665 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:39.046649    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:39.046879 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:39.046756    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:39.901864 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.901676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:39.902524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.901968    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:02:39.902524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:39.902140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-msf7f"
Jul 28 19:02:40.250056 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.249978    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="94e0ec8e1191a533c2f5bcd12af62d61cb0a6a7bd8730bdaf76717fa710ef169" exitCode=0
Jul 28 19:02:40.250376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.250066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"94e0ec8e1191a533c2f5bcd12af62d61cb0a6a7bd8730bdaf76717fa710ef169"}
Jul 28 19:02:40.253724 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.253700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" event={"ID":"4eac62bc-bac0-4579-b519-8e202835eadf","Type":"ContainerStarted","Data":"bc96cee9ee7fdda42401aa8bb9a66bb6d0745f712adaebbe9c9bfe5999897cce"}
Jul 28 19:02:40.253954 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.253936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:40.254080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.254061    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:40.254154 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.254134    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:40.269357 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.269336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:40.269472 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.269430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:02:40.298163 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:40.298123    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp" podStartSLOduration=7.908566948 podStartE2EDuration="27.298109693s" totalImagesPullingTime="19.389542745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:40.297057053 +0000 UTC m=+27.738977561" watchObservedRunningTime="2026-07-28 19:02:40.298109693 +0000 UTC m=+27.740030199"
Jul 28 19:02:41.049149 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.049130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:41.049491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.049229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:41.049491 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.049235    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:41.049491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.049244    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:41.049491 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.049325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:41.049491 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.049394    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:41.655861 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.655746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k5lrq"]
Jul 28 19:02:41.655861 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.655867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:41.656190 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.655978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:41.659250 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.659141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5g9jv"]
Jul 28 19:02:41.659385 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.659255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:41.659385 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.659361    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:41.659788 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.659769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xpd7r"]
Jul 28 19:02:41.659860 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:41.659832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:41.659939 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:41.659918    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:42.259321 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:42.259290    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="7a3c4b412955137af08cebd59990a69f020e734075bf46152b8fa07525d3b1bc" exitCode=0
Jul 28 19:02:42.259677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:42.259384    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"7a3c4b412955137af08cebd59990a69f020e734075bf46152b8fa07525d3b1bc"}
Jul 28 19:02:43.046562 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:43.046528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:43.046747 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:43.046597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:43.046747 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:43.046622    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:43.046747 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:43.046699    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:44.046267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:44.046160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:44.046851 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:44.046347    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:44.265380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:44.265343    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="b99e8ed25d384a57afabd0630d5c2f0f690afec0e26f97fba8d82dd444cb8ac8" exitCode=0
Jul 28 19:02:44.265530 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:44.265385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"b99e8ed25d384a57afabd0630d5c2f0f690afec0e26f97fba8d82dd444cb8ac8"}
Jul 28 19:02:45.046012 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:45.045976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:45.046192 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:45.046120    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k5lrq" podUID="c78e11fe-9cdb-4d9f-9943-ace27f6ea549"
Jul 28 19:02:45.046192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:45.046172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:45.046642 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:45.046311    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5g9jv" podUID="ea2fbe2e-e91e-416c-89d5-2f621a27990b"
Jul 28 19:02:46.046543 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.046513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:46.047047 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.046637    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xpd7r" podUID="5d7dbfa2-ba94-4448-abde-fd103f8a4f03"
Jul 28 19:02:46.395462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.395435    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-99.ec2.internal" event="NodeReady"
Jul 28 19:02:46.395657 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.395581    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:02:46.434577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.434551    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:02:46.436619 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.436598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.441239 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.441014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:02:46.441239 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.441145    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:02:46.441369 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.441282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:02:46.441417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.441388    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcrsr\""
Jul 28 19:02:46.446272 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.446253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:02:46.452766 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.452748    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9tx22"]
Jul 28 19:02:46.454409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.454391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.456803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.456783    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:02:46.457237 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.457049    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:02:46.457632 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.457610    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:02:46.459946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.458718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:02:46.459946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.459477    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9trb9"]
Jul 28 19:02:46.461799 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.461781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.465058 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.465005    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:02:46.465556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.465344    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:02:46.465556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.465409    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nww2p\""
Jul 28 19:02:46.465556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.465466    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:02:46.471892 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.471864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tx22"]
Jul 28 19:02:46.479097 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.479077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9trb9"]
Jul 28 19:02:46.528734 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.528880 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zthk\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.528880 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.528880 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.529080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.529080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.528950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.529080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.529022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.529080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.529060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.615098 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.615062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:46.629926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.629893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.629937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f72dff-7235-4d29-a235-e05baa75c94e-config-volume\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.630071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.629976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.629998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.630071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:46.630071 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-cert\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p677g\" (UniqueName: \"kubernetes.io/projected/c2f72dff-7235-4d29-a235-e05baa75c94e-kube-api-access-p677g\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.630354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f72dff-7235-4d29-a235-e05baa75c94e-tmp-dir\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630355    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630372    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630386    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630432    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret podName:c78e11fe-9cdb-4d9f-9943-ace27f6ea549 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:18.630411714 +0000 UTC m=+66.072332217 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret") pod "global-pull-secret-syncer-k5lrq" (UID: "c78e11fe-9cdb-4d9f-9943-ace27f6ea549") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630467    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:47.130450123 +0000 UTC m=+34.572370629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630553    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630567    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630579    2577 projected.go:196] Error preparing data for projected volume kube-api-access-pzw8s for pod openshift-network-diagnostics/network-check-target-xpd7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.630615    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s podName:5d7dbfa2-ba94-4448-abde-fd103f8a4f03 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:18.630602981 +0000 UTC m=+66.072523474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pzw8s" (UniqueName: "kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s") pod "network-check-target-xpd7r" (UID: "5d7dbfa2-ba94-4448-abde-fd103f8a4f03") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:46.630732 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.631266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.631266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zthk\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.631266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjgxr\" (UniqueName: \"kubernetes.io/projected/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-kube-api-access-jjgxr\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.631266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.630890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.631557 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.631530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.634616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.634598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.634704 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.634595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.640820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.640796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zthk\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.641278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.641258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:46.731508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjgxr\" (UniqueName: \"kubernetes.io/projected/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-kube-api-access-jjgxr\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.731508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:46.731508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f72dff-7235-4d29-a235-e05baa75c94e-config-volume\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.731764 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.731616    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:46.731764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.731764 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.731699    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:03:18.731678302 +0000 UTC m=+66.173598808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:46.731764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-cert\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.731764 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.731746    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:02:46.732022 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p677g\" (UniqueName: \"kubernetes.io/projected/c2f72dff-7235-4d29-a235-e05baa75c94e-kube-api-access-p677g\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.732022 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.731806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f72dff-7235-4d29-a235-e05baa75c94e-tmp-dir\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.732022 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:46.731818    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:02:47.23180176 +0000 UTC m=+34.673722268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:02:46.732170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.732072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f72dff-7235-4d29-a235-e05baa75c94e-tmp-dir\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.732170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.732118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2f72dff-7235-4d29-a235-e05baa75c94e-config-volume\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.734514 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.734489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-cert\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.742227 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.742168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p677g\" (UniqueName: \"kubernetes.io/projected/c2f72dff-7235-4d29-a235-e05baa75c94e-kube-api-access-p677g\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:46.742436 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.742414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjgxr\" (UniqueName: \"kubernetes.io/projected/560782e8-1ff7-4fa5-b66c-fe5ee1e1f176-kube-api-access-jjgxr\") pod \"ingress-canary-9trb9\" (UID: \"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176\") " pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.773836 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.773804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9trb9"
Jul 28 19:02:46.919953 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:46.919718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9trb9"]
Jul 28 19:02:47.046569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.046469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:02:47.046569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.046469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:02:47.054794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.054774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:02:47.055119 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.055096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:02:47.055342 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.055322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:02:47.135663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.135625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:47.135854 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:47.135802    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:02:47.135854 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:47.135823    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:02:47.135976 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:47.135915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:48.135892035 +0000 UTC m=+35.577812546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:02:47.236144 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:47.236104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:47.236350 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:47.236267    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:02:47.236350 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:47.236344    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:02:48.236325189 +0000 UTC m=+35.678245708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:02:48.046231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.046185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:02:48.048915 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.048893    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:02:48.049587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.049566    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:02:48.049699 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.049624    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:02:48.145107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.145069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:48.145329 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:48.145241    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:02:48.145329 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:48.145262    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:02:48.145329 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:48.145328    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:50.145307405 +0000 UTC m=+37.587227913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:02:48.246018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.245976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:48.246170 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:48.246153    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:02:48.246254 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:48.246239    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:02:50.246205454 +0000 UTC m=+37.688125943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:02:48.324630 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:02:48.324564    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560782e8_1ff7_4fa5_b66c_fe5ee1e1f176.slice/crio-5dd3d07f3cdd106caaa444c1f10a057eb6302a804defa93e92fbf13087fd1b73 WatchSource:0}: Error finding container 5dd3d07f3cdd106caaa444c1f10a057eb6302a804defa93e92fbf13087fd1b73: Status 404 returned error can't find the container with id 5dd3d07f3cdd106caaa444c1f10a057eb6302a804defa93e92fbf13087fd1b73
Jul 28 19:02:48.348343 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:48.348317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:49.280284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:49.280250    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="e4804f86bfd13b47c8b185f1f26f9f2184570d8f118aeffa045240d6588b6291" exitCode=0
Jul 28 19:02:49.280749 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:49.280311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"e4804f86bfd13b47c8b185f1f26f9f2184570d8f118aeffa045240d6588b6291"}
Jul 28 19:02:49.281710 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:49.281681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9trb9" event={"ID":"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176","Type":"ContainerStarted","Data":"5dd3d07f3cdd106caaa444c1f10a057eb6302a804defa93e92fbf13087fd1b73"}
Jul 28 19:02:50.161503 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:50.161466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:50.161704 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:50.161650    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:02:50.161704 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:50.161673    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:02:50.161818 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:50.161746    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:54.161724101 +0000 UTC m=+41.603644607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:02:50.262326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:50.262290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:50.262510 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:50.262449    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:02:50.262565 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:50.262516    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:02:54.262499347 +0000 UTC m=+41.704419836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:02:50.288317 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:50.288281    2577 generic.go:356] "Generic (PLEG): container finished" podID="40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7" containerID="6ffd00e005ac0104b3c196772d215c80b9907c0599f2520ba67ae1da301e9c30" exitCode=0
Jul 28 19:02:50.288702 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:50.288340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerDied","Data":"6ffd00e005ac0104b3c196772d215c80b9907c0599f2520ba67ae1da301e9c30"}
Jul 28 19:02:51.291950 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.291727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9trb9" event={"ID":"560782e8-1ff7-4fa5-b66c-fe5ee1e1f176","Type":"ContainerStarted","Data":"b68269428b9e88c4d6878bfdb2ec53ac21d68564d992d603b33a785963b18d1c"}
Jul 28 19:02:51.294905 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.294882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" event={"ID":"40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7","Type":"ContainerStarted","Data":"c97efdb5664240b35ede71c91d3b60c50c7f61b34d234fd5fcb03ca6827688e8"}
Jul 28 19:02:51.306876 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.306835    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9trb9" podStartSLOduration=2.966445199 podStartE2EDuration="5.306819565s" totalImagesPullingTime="2.340374366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:51.30648109 +0000 UTC m=+38.748401599" watchObservedRunningTime="2026-07-28 19:02:51.306819565 +0000 UTC m=+38.748740073"
Jul 28 19:02:51.328666 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.328628    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tq6kq" podStartSLOduration=8.080579832 podStartE2EDuration="38.328614334s" totalImagesPullingTime="30.248034502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:51.328450358 +0000 UTC m=+38.770370866" watchObservedRunningTime="2026-07-28 19:02:51.328614334 +0000 UTC m=+38.770534842"
Jul 28 19:02:51.457950 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.457872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:51.514377 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:51.514352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:52.161761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:52.161725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:52.227403 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:52.227379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:54.185698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:54.185666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:02:54.186066 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:54.185786    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:02:54.186066 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:54.185802    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:02:54.186066 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:54.185851    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:02.185837964 +0000 UTC m=+49.627758450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:02:54.286681 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:02:54.286651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:02:54.286817 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:54.286795    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:02:54.286872 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:02:54.286862    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:02.286846118 +0000 UTC m=+49.728766605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:02.240012 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:02.239975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:02.240466 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:02.240084    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:02.240466 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:02.240095    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:03:02.240466 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:02.240151    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:18.240137375 +0000 UTC m=+65.682057862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:03:02.341306 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:02.341271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:02.341455 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:02.341407    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:02.341503 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:02.341492    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:18.341479346 +0000 UTC m=+65.783399835 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:12.276896 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:12.276866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd2xp"
Jul 28 19:03:18.247326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.247278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:18.247788 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.247417    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:18.247788 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.247435    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db948c5b7-cpw58: secret "image-registry-tls" not found
Jul 28 19:03:18.247788 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.247493    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls podName:7c2ba24f-5a21-4010-8024-3f363c23a8a5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:50.247474752 +0000 UTC m=+97.689395239 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls") pod "image-registry-db948c5b7-cpw58" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5") : secret "image-registry-tls" not found
Jul 28 19:03:18.347955 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.347921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:18.348091 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.348069    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:18.348145 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.348136    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls podName:c2f72dff-7235-4d29-a235-e05baa75c94e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:50.348120027 +0000 UTC m=+97.790040512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls") pod "dns-default-9tx22" (UID: "c2f72dff-7235-4d29-a235-e05baa75c94e") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:18.650632 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.650602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:03:18.650797 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.650641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:03:18.656715 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.656689    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:03:18.657084 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.657067    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:03:18.663604 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.663587    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:03:18.664797 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.664779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c78e11fe-9cdb-4d9f-9943-ace27f6ea549-original-pull-secret\") pod \"global-pull-secret-syncer-k5lrq\" (UID: \"c78e11fe-9cdb-4d9f-9943-ace27f6ea549\") " pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:03:18.673949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.673931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzw8s\" (UniqueName: \"kubernetes.io/projected/5d7dbfa2-ba94-4448-abde-fd103f8a4f03-kube-api-access-pzw8s\") pod \"network-check-target-xpd7r\" (UID: \"5d7dbfa2-ba94-4448-abde-fd103f8a4f03\") " pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:03:18.751697 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.751662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:03:18.754350 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.754332    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:03:18.762479 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.762460    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 19:03:18.762532 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:18.762517    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs podName:ea2fbe2e-e91e-416c-89d5-2f621a27990b nodeName:}" failed. No retries permitted until 2026-07-28 19:04:22.762502456 +0000 UTC m=+130.204422942 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs") pod "network-metrics-daemon-5g9jv" (UID: "ea2fbe2e-e91e-416c-89d5-2f621a27990b") : secret "metrics-daemon-secret" not found
Jul 28 19:03:18.864282 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.864237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k5lrq"
Jul 28 19:03:18.965724 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.965687    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:03:18.966862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.966796    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:03:18.998002 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:18.997969    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k5lrq"]
Jul 28 19:03:19.001369 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:19.001342    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78e11fe_9cdb_4d9f_9943_ace27f6ea549.slice/crio-33a04c657a61e9aad08b45beda367fb6f26c3be548bc4a5ac4155e562963df36 WatchSource:0}: Error finding container 33a04c657a61e9aad08b45beda367fb6f26c3be548bc4a5ac4155e562963df36: Status 404 returned error can't find the container with id 33a04c657a61e9aad08b45beda367fb6f26c3be548bc4a5ac4155e562963df36
Jul 28 19:03:19.087571 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:19.087538    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xpd7r"]
Jul 28 19:03:19.090074 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:19.090045    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7dbfa2_ba94_4448_abde_fd103f8a4f03.slice/crio-5fd41f4869343d340adf74210c8f7d2347e866db876460a0b9492edd3e39aa67 WatchSource:0}: Error finding container 5fd41f4869343d340adf74210c8f7d2347e866db876460a0b9492edd3e39aa67: Status 404 returned error can't find the container with id 5fd41f4869343d340adf74210c8f7d2347e866db876460a0b9492edd3e39aa67
Jul 28 19:03:19.357000 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:19.356962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xpd7r" event={"ID":"5d7dbfa2-ba94-4448-abde-fd103f8a4f03","Type":"ContainerStarted","Data":"5fd41f4869343d340adf74210c8f7d2347e866db876460a0b9492edd3e39aa67"}
Jul 28 19:03:19.357902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:19.357881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k5lrq" event={"ID":"c78e11fe-9cdb-4d9f-9943-ace27f6ea549","Type":"ContainerStarted","Data":"33a04c657a61e9aad08b45beda367fb6f26c3be548bc4a5ac4155e562963df36"}
Jul 28 19:03:23.993846 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:23.993663    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"]
Jul 28 19:03:23.996468 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:23.996453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"
Jul 28 19:03:23.999578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:23.999557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rsbc8\""
Jul 28 19:03:24.020926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.020904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"]
Jul 28 19:03:24.086129 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.086095    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"]
Jul 28 19:03:24.088975 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.088960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.091284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.091260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b6lh\" (UniqueName: \"kubernetes.io/projected/cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb-kube-api-access-6b6lh\") pod \"network-check-source-7d88cb55d8-c8svl\" (UID: \"cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"
Jul 28 19:03:24.092380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.092361    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 19:03:24.092558 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.092540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.092627 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.092557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qxwdq\""
Jul 28 19:03:24.092627 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.092582    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"]
Jul 28 19:03:24.093427 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.093412    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.095268 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.095245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.098904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.098752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.104340 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.104295    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7rsr2\""
Jul 28 19:03:24.105080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.105064    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"]
Jul 28 19:03:24.109236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.109203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"]
Jul 28 19:03:24.110336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.110319    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 19:03:24.114595 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.114578    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.120564 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.120550    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 19:03:24.192116 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.192232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77642228-1e12-416d-9ddf-a8168eb4823e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.192232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhr7h\" (UniqueName: \"kubernetes.io/projected/c69fa32d-5537-4a4c-8953-607a878672f6-kube-api-access-bhr7h\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.192232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkdq\" (UniqueName: \"kubernetes.io/projected/77642228-1e12-416d-9ddf-a8168eb4823e-kube-api-access-knkdq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.192337 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b6lh\" (UniqueName: \"kubernetes.io/projected/cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb-kube-api-access-6b6lh\") pod \"network-check-source-7d88cb55d8-c8svl\" (UID: \"cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"
Jul 28 19:03:24.192337 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.192311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77642228-1e12-416d-9ddf-a8168eb4823e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.215377 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.215356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b6lh\" (UniqueName: \"kubernetes.io/projected/cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb-kube-api-access-6b6lh\") pod \"network-check-source-7d88cb55d8-c8svl\" (UID: \"cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"
Jul 28 19:03:24.293233 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77642228-1e12-416d-9ddf-a8168eb4823e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.293233 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.293406 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77642228-1e12-416d-9ddf-a8168eb4823e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.293406 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:24.293344    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:03:24.293505 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:24.293415    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls podName:c69fa32d-5537-4a4c-8953-607a878672f6 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.793394964 +0000 UTC m=+72.235315454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls") pod "cluster-samples-operator-6748489656-m8vg5" (UID: "c69fa32d-5537-4a4c-8953-607a878672f6") : secret "samples-operator-tls" not found
Jul 28 19:03:24.293505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhr7h\" (UniqueName: \"kubernetes.io/projected/c69fa32d-5537-4a4c-8953-607a878672f6-kube-api-access-bhr7h\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.293505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knkdq\" (UniqueName: \"kubernetes.io/projected/77642228-1e12-416d-9ddf-a8168eb4823e-kube-api-access-knkdq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.293939 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.293917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77642228-1e12-416d-9ddf-a8168eb4823e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.295441 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.295424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77642228-1e12-416d-9ddf-a8168eb4823e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.302157 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.302134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhr7h\" (UniqueName: \"kubernetes.io/projected/c69fa32d-5537-4a4c-8953-607a878672f6-kube-api-access-bhr7h\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.302379 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.302363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkdq\" (UniqueName: \"kubernetes.io/projected/77642228-1e12-416d-9ddf-a8168eb4823e-kube-api-access-knkdq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh\" (UID: \"77642228-1e12-416d-9ddf-a8168eb4823e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.304232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.304207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"
Jul 28 19:03:24.377199 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.377136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k5lrq" event={"ID":"c78e11fe-9cdb-4d9f-9943-ace27f6ea549","Type":"ContainerStarted","Data":"76c44940112d86a40e0b2c961eb1b3a5773ebfecf273d034a4972f60c2a7f043"}
Jul 28 19:03:24.379841 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.379808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xpd7r" event={"ID":"5d7dbfa2-ba94-4448-abde-fd103f8a4f03","Type":"ContainerStarted","Data":"46d37b46a05cf98b43b3a1221ebfda41a334bf67f978959ecc082915e0f34ea7"}
Jul 28 19:03:24.380187 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.380169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:03:24.397555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.397514    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k5lrq" podStartSLOduration=66.672547972 podStartE2EDuration="1m11.397500689s" totalImagesPullingTime="4.724952717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:24.396311789 +0000 UTC m=+71.838232298" watchObservedRunningTime="2026-07-28 19:03:24.397500689 +0000 UTC m=+71.839421196"
Jul 28 19:03:24.405275 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.405256    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"
Jul 28 19:03:24.411746 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.411712    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xpd7r" podStartSLOduration=66.779367399 podStartE2EDuration="1m11.411701358s" totalImagesPullingTime="4.632333959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:24.411112917 +0000 UTC m=+71.853033436" watchObservedRunningTime="2026-07-28 19:03:24.411701358 +0000 UTC m=+71.853621865"
Jul 28 19:03:24.421099 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.421080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl"]
Jul 28 19:03:24.423883 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:24.423864    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4d1300_24b7_4ea2_bdb2_aa8cb82c21fb.slice/crio-8e113a54d4ac656202427d46ae605d8855aba66785b547752480a8231430f4c6 WatchSource:0}: Error finding container 8e113a54d4ac656202427d46ae605d8855aba66785b547752480a8231430f4c6: Status 404 returned error can't find the container with id 8e113a54d4ac656202427d46ae605d8855aba66785b547752480a8231430f4c6
Jul 28 19:03:24.528205 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.528172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh"]
Jul 28 19:03:24.531355 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:24.531334    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77642228_1e12_416d_9ddf_a8168eb4823e.slice/crio-c2bbf03f14ce3570c11f247be338263dac5202e14d28051ff700e62a916dcb70 WatchSource:0}: Error finding container c2bbf03f14ce3570c11f247be338263dac5202e14d28051ff700e62a916dcb70: Status 404 returned error can't find the container with id c2bbf03f14ce3570c11f247be338263dac5202e14d28051ff700e62a916dcb70
Jul 28 19:03:24.797330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:24.797302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:24.797508 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:24.797425    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:03:24.797508 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:24.797493    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls podName:c69fa32d-5537-4a4c-8953-607a878672f6 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.7974742 +0000 UTC m=+73.239394702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls") pod "cluster-samples-operator-6748489656-m8vg5" (UID: "c69fa32d-5537-4a4c-8953-607a878672f6") : secret "samples-operator-tls" not found
Jul 28 19:03:25.383826 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:25.383781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh" event={"ID":"77642228-1e12-416d-9ddf-a8168eb4823e","Type":"ContainerStarted","Data":"c2bbf03f14ce3570c11f247be338263dac5202e14d28051ff700e62a916dcb70"}
Jul 28 19:03:25.385227 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:25.385175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl" event={"ID":"cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb","Type":"ContainerStarted","Data":"62bad1fd5983c53a79ad25b050fdd4c63a1bba75907b8251da8e81b24659c48c"}
Jul 28 19:03:25.385351 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:25.385244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl" event={"ID":"cc4d1300-24b7-4ea2-bdb2-aa8cb82c21fb","Type":"ContainerStarted","Data":"8e113a54d4ac656202427d46ae605d8855aba66785b547752480a8231430f4c6"}
Jul 28 19:03:25.399906 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:25.399859    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c8svl" podStartSLOduration=2.399843971 podStartE2EDuration="2.399843971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:25.399266002 +0000 UTC m=+72.841186510" watchObservedRunningTime="2026-07-28 19:03:25.399843971 +0000 UTC m=+72.841764481"
Jul 28 19:03:25.804505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:25.804418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:25.804657 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:25.804578    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:03:25.804708 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:25.804658    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls podName:c69fa32d-5537-4a4c-8953-607a878672f6 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.804637228 +0000 UTC m=+75.246557714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls") pod "cluster-samples-operator-6748489656-m8vg5" (UID: "c69fa32d-5537-4a4c-8953-607a878672f6") : secret "samples-operator-tls" not found
Jul 28 19:03:27.391952 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:27.391910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh" event={"ID":"77642228-1e12-416d-9ddf-a8168eb4823e","Type":"ContainerStarted","Data":"ee0da683bd3aa426e163c2209e96d53074212173a3838a04b9aeee693dcd635d"}
Jul 28 19:03:27.411110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:27.411039    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh" podStartSLOduration=1.062038844 podStartE2EDuration="3.411023898s" totalImagesPullingTime="2.348985054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:27.410359912 +0000 UTC m=+74.852280419" watchObservedRunningTime="2026-07-28 19:03:27.411023898 +0000 UTC m=+74.852944384"
Jul 28 19:03:27.819361 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:27.819265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:27.819502 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:27.819408    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:03:27.819502 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:27.819472    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls podName:c69fa32d-5537-4a4c-8953-607a878672f6 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.819455455 +0000 UTC m=+79.261375941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls") pod "cluster-samples-operator-6748489656-m8vg5" (UID: "c69fa32d-5537-4a4c-8953-607a878672f6") : secret "samples-operator-tls" not found
Jul 28 19:03:28.492486 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.492448    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jjvj7"]
Jul 28 19:03:28.495567 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.495551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.497870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.497850    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:03:28.498329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.498317    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:03:28.498877 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.498861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:03:28.498927 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.498877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26hct\""
Jul 28 19:03:28.499185 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.499167    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:03:28.518633 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.518611    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjvj7"]
Jul 28 19:03:28.625470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.625439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.625470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.625474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xmgw\" (UniqueName: \"kubernetes.io/projected/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-api-access-4xmgw\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.625628 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.625502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a075b9da-2afb-47fa-ac98-f30aac172c3e-data-volume\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.625628 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.625563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a075b9da-2afb-47fa-ac98-f30aac172c3e-crio-socket\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.625628 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.625612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.726740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.726711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.726740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.726739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xmgw\" (UniqueName: \"kubernetes.io/projected/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-api-access-4xmgw\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.726942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.726767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a075b9da-2afb-47fa-ac98-f30aac172c3e-data-volume\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.726942 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:28.726833    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.726942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.726898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a075b9da-2afb-47fa-ac98-f30aac172c3e-crio-socket\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.726942 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:28.726908    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.226889626 +0000 UTC m=+76.668810139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.726942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.726934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.727114 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.727016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a075b9da-2afb-47fa-ac98-f30aac172c3e-crio-socket\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.727114 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:28.727032    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.727114 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:28.727072    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.22706101 +0000 UTC m=+76.668981495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.727114 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.727079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a075b9da-2afb-47fa-ac98-f30aac172c3e-data-volume\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:28.734779 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:28.734760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xmgw\" (UniqueName: \"kubernetes.io/projected/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-api-access-4xmgw\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:29.230113 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:29.230065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:29.230343 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:29.230241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:29.230343 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:29.230258    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.230343 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:29.230309    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:29.230343 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:29.230338    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.230317694 +0000 UTC m=+77.672238180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.230561 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:29.230359    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.230349373 +0000 UTC m=+77.672269862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.235987 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:30.235938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:30.236409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:30.235999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:30.236409 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:30.236075    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.236409 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:30.236091    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.236409 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:30.236146    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.23612969 +0000 UTC m=+79.678050176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.236409 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:30.236159    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.236153717 +0000 UTC m=+79.678074202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.264296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:30.264267    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2gt4r_34a1b57d-0dca-4762-97af-5cff080bacde/dns-node-resolver/0.log"
Jul 28 19:03:31.264764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.264739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8m52c_bdaf7e7e-023f-494c-9126-284b229cdf69/node-ca/0.log"
Jul 28 19:03:31.585863 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.585827    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7nvxq"]
Jul 28 19:03:31.589147 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.589125    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.593304 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.593282    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 19:03:31.593415 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.593355    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 19:03:31.593415 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.593364    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 19:03:31.594250 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.594236    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rvkcs\""
Jul 28 19:03:31.594250 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.594244    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 19:03:31.599031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.599012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7nvxq"]
Jul 28 19:03:31.645767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.645743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms6bd\" (UniqueName: \"kubernetes.io/projected/960f1c36-05b9-4fef-8379-125142210496-kube-api-access-ms6bd\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.645861 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.645844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/960f1c36-05b9-4fef-8379-125142210496-signing-cabundle\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.645918 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.645904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/960f1c36-05b9-4fef-8379-125142210496-signing-key\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.746592 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.746568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/960f1c36-05b9-4fef-8379-125142210496-signing-cabundle\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.746677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.746622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/960f1c36-05b9-4fef-8379-125142210496-signing-key\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.746677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.746649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms6bd\" (UniqueName: \"kubernetes.io/projected/960f1c36-05b9-4fef-8379-125142210496-kube-api-access-ms6bd\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.747181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.747161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/960f1c36-05b9-4fef-8379-125142210496-signing-cabundle\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.748925 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.748907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/960f1c36-05b9-4fef-8379-125142210496-signing-key\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.754599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.754573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms6bd\" (UniqueName: \"kubernetes.io/projected/960f1c36-05b9-4fef-8379-125142210496-kube-api-access-ms6bd\") pod \"service-ca-658d7b7c78-7nvxq\" (UID: \"960f1c36-05b9-4fef-8379-125142210496\") " pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:31.847325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.847278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:31.847409 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:31.847397    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:03:31.847459 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:31.847449    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls podName:c69fa32d-5537-4a4c-8953-607a878672f6 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.847432717 +0000 UTC m=+87.289353204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls") pod "cluster-samples-operator-6748489656-m8vg5" (UID: "c69fa32d-5537-4a4c-8953-607a878672f6") : secret "samples-operator-tls" not found
Jul 28 19:03:31.898074 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:31.898054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq"
Jul 28 19:03:32.020286 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:32.020259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7nvxq"]
Jul 28 19:03:32.023362 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:32.023339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960f1c36_05b9_4fef_8379_125142210496.slice/crio-69562fede104b895592f68a2672e4e46f68284b7a661fb2aa6e220acb10cf39b WatchSource:0}: Error finding container 69562fede104b895592f68a2672e4e46f68284b7a661fb2aa6e220acb10cf39b: Status 404 returned error can't find the container with id 69562fede104b895592f68a2672e4e46f68284b7a661fb2aa6e220acb10cf39b
Jul 28 19:03:32.064359 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:32.064336    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9trb9_560782e8-1ff7-4fa5-b66c-fe5ee1e1f176/serve-healthcheck-canary/0.log"
Jul 28 19:03:32.250310 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:32.250245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:32.250310 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:32.250304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:32.250440 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:32.250389    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.250440 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:32.250401    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.250440 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:32.250438    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.250425215 +0000 UTC m=+83.692345702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.250541 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:32.250450    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.250444265 +0000 UTC m=+83.692364752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.405186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:32.405156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq" event={"ID":"960f1c36-05b9-4fef-8379-125142210496","Type":"ContainerStarted","Data":"69562fede104b895592f68a2672e4e46f68284b7a661fb2aa6e220acb10cf39b"}
Jul 28 19:03:34.412157 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:34.412112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq" event={"ID":"960f1c36-05b9-4fef-8379-125142210496","Type":"ContainerStarted","Data":"55484aba22dd21dd119bba09a6ba451baef4684c431d599dc27f841c431a38f7"}
Jul 28 19:03:34.435765 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:34.435717    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-7nvxq" podStartSLOduration=1.186902794 podStartE2EDuration="3.435702365s" totalImagesPullingTime="2.248799571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:34.434701456 +0000 UTC m=+81.876621961" watchObservedRunningTime="2026-07-28 19:03:34.435702365 +0000 UTC m=+81.877622872"
Jul 28 19:03:36.283609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:36.283476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:36.283609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:36.283541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:36.284124 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:36.283606    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.284124 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:36.283646    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:36.284124 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:36.283687    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.283666582 +0000 UTC m=+91.725587067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.284124 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:36.283707    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.283698105 +0000 UTC m=+91.725618593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:39.914245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:39.914186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:39.916647 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:39.916622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69fa32d-5537-4a4c-8953-607a878672f6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m8vg5\" (UID: \"c69fa32d-5537-4a4c-8953-607a878672f6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:39.997855 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:39.997829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"
Jul 28 19:03:40.122439 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:40.122408    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5"]
Jul 28 19:03:40.432225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:40.432134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5" event={"ID":"c69fa32d-5537-4a4c-8953-607a878672f6","Type":"ContainerStarted","Data":"a0144d522a97aba45c89cd763abffad640b8aacaf4281bc535be2fea87fc35de"}
Jul 28 19:03:42.439517 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:42.439487    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5" event={"ID":"c69fa32d-5537-4a4c-8953-607a878672f6","Type":"ContainerStarted","Data":"acef065d98b5c4359fb484ce4310ada035ad6796b1edfdaafdbc097f29537421"}
Jul 28 19:03:42.439907 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:42.439535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5" event={"ID":"c69fa32d-5537-4a4c-8953-607a878672f6","Type":"ContainerStarted","Data":"b59318e18a78c3cceac6ed66500f144f9e991567be7e03a7e65783d7740e49cd"}
Jul 28 19:03:42.456001 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:42.455936    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m8vg5" podStartSLOduration=16.836431925 podStartE2EDuration="18.455918049s" totalImagesPullingTime="1.619486124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:42.455506 +0000 UTC m=+89.897426507" watchObservedRunningTime="2026-07-28 19:03:42.455918049 +0000 UTC m=+89.897838558"
Jul 28 19:03:44.350876 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:44.350833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:44.351366 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:44.350917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:44.351366 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:44.351060    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.351366 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:03:44.351142    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls podName:a075b9da-2afb-47fa-ac98-f30aac172c3e nodeName:}" failed. No retries permitted until 2026-07-28 19:04:00.351122203 +0000 UTC m=+107.793042704 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jjvj7" (UID: "a075b9da-2afb-47fa-ac98-f30aac172c3e") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.351366 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:44.351356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a075b9da-2afb-47fa-ac98-f30aac172c3e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:03:50.297662 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.297631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:50.299714 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.299693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"image-registry-db948c5b7-cpw58\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") " pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:50.351308 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.351268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcrsr\""
Jul 28 19:03:50.359422 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.359400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:50.398478 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.398447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:50.400497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.400471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2f72dff-7235-4d29-a235-e05baa75c94e-metrics-tls\") pod \"dns-default-9tx22\" (UID: \"c2f72dff-7235-4d29-a235-e05baa75c94e\") " pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:50.480273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.480234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:03:50.484863 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:50.484834    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2ba24f_5a21_4010_8024_3f363c23a8a5.slice/crio-7e9de5dc85026050b1b8295fbb3c4a594d4983c7933c19225477f5be5307b1cb WatchSource:0}: Error finding container 7e9de5dc85026050b1b8295fbb3c4a594d4983c7933c19225477f5be5307b1cb: Status 404 returned error can't find the container with id 7e9de5dc85026050b1b8295fbb3c4a594d4983c7933c19225477f5be5307b1cb
Jul 28 19:03:50.667722 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.667695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:03:50.676665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.676641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:50.796174 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:50.796145    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tx22"]
Jul 28 19:03:50.799377 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:50.799354    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f72dff_7235_4d29_a235_e05baa75c94e.slice/crio-ee1ae7f24fbc2f367d41ea93d803bfc4269ed920e9b8d6d950f72ee77d5fe63e WatchSource:0}: Error finding container ee1ae7f24fbc2f367d41ea93d803bfc4269ed920e9b8d6d950f72ee77d5fe63e: Status 404 returned error can't find the container with id ee1ae7f24fbc2f367d41ea93d803bfc4269ed920e9b8d6d950f72ee77d5fe63e
Jul 28 19:03:51.466851 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:51.466768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tx22" event={"ID":"c2f72dff-7235-4d29-a235-e05baa75c94e","Type":"ContainerStarted","Data":"ee1ae7f24fbc2f367d41ea93d803bfc4269ed920e9b8d6d950f72ee77d5fe63e"}
Jul 28 19:03:51.468513 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:51.468486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" event={"ID":"7c2ba24f-5a21-4010-8024-3f363c23a8a5","Type":"ContainerStarted","Data":"bca4c8f722cef9c3c090d104904ac69af8d9d6d21f56bd251ccd8945230d0ac1"}
Jul 28 19:03:51.468625 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:51.468528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" event={"ID":"7c2ba24f-5a21-4010-8024-3f363c23a8a5","Type":"ContainerStarted","Data":"7e9de5dc85026050b1b8295fbb3c4a594d4983c7933c19225477f5be5307b1cb"}
Jul 28 19:03:51.468673 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:51.468621    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:03:51.490585 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:51.490533    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" podStartSLOduration=98.490517064 podStartE2EDuration="1m38.490517064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:51.490135268 +0000 UTC m=+98.932055777" watchObservedRunningTime="2026-07-28 19:03:51.490517064 +0000 UTC m=+98.932437573"
Jul 28 19:03:52.473301 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:52.473201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tx22" event={"ID":"c2f72dff-7235-4d29-a235-e05baa75c94e","Type":"ContainerStarted","Data":"273708b64113e74430fe97e2162a2641c41831af771dd9a6aaf3fb8c9a2326f1"}
Jul 28 19:03:52.473301 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:52.473279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tx22" event={"ID":"c2f72dff-7235-4d29-a235-e05baa75c94e","Type":"ContainerStarted","Data":"6f4f3ed6f9ea962e47178af4694e1376adcac9992fd2c80fc14c6fc25b697aba"}
Jul 28 19:03:52.491017 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:52.490973    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9tx22" podStartSLOduration=65.134328314 podStartE2EDuration="1m6.49095904s" totalImagesPullingTime="1.356630726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:52.490446687 +0000 UTC m=+99.932367205" watchObservedRunningTime="2026-07-28 19:03:52.49095904 +0000 UTC m=+99.932879548"
Jul 28 19:03:53.476179 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:53.476151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9tx22"
Jul 28 19:03:54.921614 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.921583    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"]
Jul 28 19:03:54.944684 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.944658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"]
Jul 28 19:03:54.944823 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.944743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:54.947171 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.947149    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 19:03:54.948033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.948016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 19:03:54.948107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.948042    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 19:03:54.948156 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:54.948105    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 19:03:55.033922 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.033892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-tmp\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.034098 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.033966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.034098 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.034018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgqkq\" (UniqueName: \"kubernetes.io/projected/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-kube-api-access-lgqkq\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.134852 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.134816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.135038 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.134885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqkq\" (UniqueName: \"kubernetes.io/projected/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-kube-api-access-lgqkq\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.135038 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.134918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-tmp\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.135378 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.135354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-tmp\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.137360 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.137341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.161332 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.161308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgqkq\" (UniqueName: \"kubernetes.io/projected/a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1-kube-api-access-lgqkq\") pod \"klusterlet-addon-workmgr-7b5b8f6b89-f4mm7\" (UID: \"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.254522 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.254462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:55.388898 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.388869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"]
Jul 28 19:03:55.392036 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:03:55.391997    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c7e25f_123f_4c32_b6ba_6e5bd4684ea1.slice/crio-39ee155b655c8bb0083d1e48f8fb15cd314bff3c094672832ad282c81e5788ac WatchSource:0}: Error finding container 39ee155b655c8bb0083d1e48f8fb15cd314bff3c094672832ad282c81e5788ac: Status 404 returned error can't find the container with id 39ee155b655c8bb0083d1e48f8fb15cd314bff3c094672832ad282c81e5788ac
Jul 28 19:03:55.482775 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:55.482744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7" event={"ID":"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1","Type":"ContainerStarted","Data":"39ee155b655c8bb0083d1e48f8fb15cd314bff3c094672832ad282c81e5788ac"}
Jul 28 19:03:56.390689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:56.390349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xpd7r"
Jul 28 19:03:58.393168 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.393125    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"]
Jul 28 19:03:58.395651 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.395627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:03:58.398291 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.398089    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 19:03:58.398291 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.398089    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vqwph\""
Jul 28 19:03:58.404117 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.404049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"]
Jul 28 19:03:58.562143 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.562097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/61ff5af9-bb62-432a-80e8-9037b97ff515-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8spsz\" (UID: \"61ff5af9-bb62-432a-80e8-9037b97ff515\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:03:58.663384 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.663292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/61ff5af9-bb62-432a-80e8-9037b97ff515-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8spsz\" (UID: \"61ff5af9-bb62-432a-80e8-9037b97ff515\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:03:58.665780 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.665752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/61ff5af9-bb62-432a-80e8-9037b97ff515-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-8spsz\" (UID: \"61ff5af9-bb62-432a-80e8-9037b97ff515\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:03:58.706803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:58.706767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:03:59.073238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.071562    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"]
Jul 28 19:03:59.500791 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.500753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7" event={"ID":"a3c7e25f-123f-4c32-b6ba-6e5bd4684ea1","Type":"ContainerStarted","Data":"6f293de5868b76197fb59c998fbdd80a8184e8239fb75a0913ca7b7ca644a2c2"}
Jul 28 19:03:59.501251 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.500962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:59.502051 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.502023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz" event={"ID":"61ff5af9-bb62-432a-80e8-9037b97ff515","Type":"ContainerStarted","Data":"9a98020e2fd065bbdf7c6d1a119b3d97b58cfc7ee06d77bc7ff654e90018b72e"}
Jul 28 19:03:59.502849 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.502828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7"
Jul 28 19:03:59.517790 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:03:59.517749    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b5b8f6b89-f4mm7" podStartSLOduration=1.903961426 podStartE2EDuration="5.517736506s" totalImagesPullingTime="3.61377508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:59.516744467 +0000 UTC m=+106.958664973" watchObservedRunningTime="2026-07-28 19:03:59.517736506 +0000 UTC m=+106.959657010"
Jul 28 19:04:00.377291 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.377267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:04:00.379506 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.379484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a075b9da-2afb-47fa-ac98-f30aac172c3e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jjvj7\" (UID: \"a075b9da-2afb-47fa-ac98-f30aac172c3e\") " pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:04:00.506205 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.506171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz" event={"ID":"61ff5af9-bb62-432a-80e8-9037b97ff515","Type":"ContainerStarted","Data":"67c5900ebdc67975c9f8dc5902566482a44314cbe96c867cf08008e65c5b81b2"}
Jul 28 19:04:00.523613 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.523571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz" podStartSLOduration=1.238569754 podStartE2EDuration="2.523556569s" totalImagesPullingTime="1.284986815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:00.522653778 +0000 UTC m=+107.964574288" watchObservedRunningTime="2026-07-28 19:04:00.523556569 +0000 UTC m=+107.965477078"
Jul 28 19:04:00.604298 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.604279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jjvj7"
Jul 28 19:04:00.726960 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:00.726935    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jjvj7"]
Jul 28 19:04:00.729365 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:00.729343    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda075b9da_2afb_47fa_ac98_f30aac172c3e.slice/crio-cd813ee8baae6cf0cffdfefbbdf4aa93e6fa4310d401f9f578ca3d1ab14e0720 WatchSource:0}: Error finding container cd813ee8baae6cf0cffdfefbbdf4aa93e6fa4310d401f9f578ca3d1ab14e0720: Status 404 returned error can't find the container with id cd813ee8baae6cf0cffdfefbbdf4aa93e6fa4310d401f9f578ca3d1ab14e0720
Jul 28 19:04:01.512852 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:01.512784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjvj7" event={"ID":"a075b9da-2afb-47fa-ac98-f30aac172c3e","Type":"ContainerStarted","Data":"cd813ee8baae6cf0cffdfefbbdf4aa93e6fa4310d401f9f578ca3d1ab14e0720"}
Jul 28 19:04:01.513496 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:01.513290    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:04:01.519337 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:01.519136    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-8spsz"
Jul 28 19:04:02.467500 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.467467    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"]
Jul 28 19:04:02.470857 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.470831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.474378 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474142    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 19:04:02.474378 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474172    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:04:02.474378 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474272    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:04:02.474623 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kks52\""
Jul 28 19:04:02.474728 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474707    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:04:02.474786 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.474751    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 19:04:02.491035 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.491010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"]
Jul 28 19:04:02.596401 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.596371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj4fv\" (UniqueName: \"kubernetes.io/projected/ae28584c-ec88-4e95-8b40-33df233b25aa-kube-api-access-kj4fv\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.596877 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.596416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.596877 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.596461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.596877 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.596543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae28584c-ec88-4e95-8b40-33df233b25aa-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.697540 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.697511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.697694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.697593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae28584c-ec88-4e95-8b40-33df233b25aa-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.697694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.697629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj4fv\" (UniqueName: \"kubernetes.io/projected/ae28584c-ec88-4e95-8b40-33df233b25aa-kube-api-access-kj4fv\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.697694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.697656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.697694 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:04:02.697682    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 28 19:04:02.697913 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:04:02.697745    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls podName:ae28584c-ec88-4e95-8b40-33df233b25aa nodeName:}" failed. No retries permitted until 2026-07-28 19:04:03.19772893 +0000 UTC m=+110.639649415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-pgvq7" (UID: "ae28584c-ec88-4e95-8b40-33df233b25aa") : secret "prometheus-operator-tls" not found
Jul 28 19:04:02.698517 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.698493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae28584c-ec88-4e95-8b40-33df233b25aa-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.700549 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.700529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:02.709053 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:02.709032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj4fv\" (UniqueName: \"kubernetes.io/projected/ae28584c-ec88-4e95-8b40-33df233b25aa-kube-api-access-kj4fv\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:03.201677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:03.201639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:03.204152 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:03.204128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae28584c-ec88-4e95-8b40-33df233b25aa-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pgvq7\" (UID: \"ae28584c-ec88-4e95-8b40-33df233b25aa\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:03.382567 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:03.382533    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"
Jul 28 19:04:03.481703 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:03.481616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9tx22"
Jul 28 19:04:05.791699 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:05.791673    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pgvq7"]
Jul 28 19:04:05.795323 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:05.795298    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae28584c_ec88_4e95_8b40_33df233b25aa.slice/crio-a8ab8c98c5b079d24fc5e803753959203d5ecdc04b0fa2041a5924d9ae6ef9e0 WatchSource:0}: Error finding container a8ab8c98c5b079d24fc5e803753959203d5ecdc04b0fa2041a5924d9ae6ef9e0: Status 404 returned error can't find the container with id a8ab8c98c5b079d24fc5e803753959203d5ecdc04b0fa2041a5924d9ae6ef9e0
Jul 28 19:04:06.532208 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:06.532164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjvj7" event={"ID":"a075b9da-2afb-47fa-ac98-f30aac172c3e","Type":"ContainerStarted","Data":"a6b57bacb13155f7a64ab1407b934b9dfe0eef2c3d2b775ee4b6e63cb88d42a9"}
Jul 28 19:04:06.533597 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:06.533498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7" event={"ID":"ae28584c-ec88-4e95-8b40-33df233b25aa","Type":"ContainerStarted","Data":"a8ab8c98c5b079d24fc5e803753959203d5ecdc04b0fa2041a5924d9ae6ef9e0"}
Jul 28 19:04:07.538890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:07.538817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7" event={"ID":"ae28584c-ec88-4e95-8b40-33df233b25aa","Type":"ContainerStarted","Data":"4a6efa40d147cd8418da34d17b3340db1fa92a50e5540c74d0302574cdde732b"}
Jul 28 19:04:07.538890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:07.538880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7" event={"ID":"ae28584c-ec88-4e95-8b40-33df233b25aa","Type":"ContainerStarted","Data":"30310111a35606b674f1e669fa212c554e80f1b3501fe92485a225ff9777490f"}
Jul 28 19:04:09.549543 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.549509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjvj7" event={"ID":"a075b9da-2afb-47fa-ac98-f30aac172c3e","Type":"ContainerStarted","Data":"19d11fe88f2daf09f08341438e7f4fa962dd8f8d3a4ef1a081d4e9eadf64cefd"}
Jul 28 19:04:09.846129 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.846068    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-pgvq7" podStartSLOduration=6.429555426 podStartE2EDuration="7.846045686s" totalImagesPullingTime="1.41649026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:07.559824498 +0000 UTC m=+115.001745006" watchObservedRunningTime="2026-07-28 19:04:09.846045686 +0000 UTC m=+117.287966196"
Jul 28 19:04:09.846325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.846295    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c2p2v"]
Jul 28 19:04:09.854189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.854164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.856683 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.856649    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:04:09.856806 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.856719    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:04:09.856806 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.856753    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kmxrs\""
Jul 28 19:04:09.857368 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.857346    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:04:09.954299 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-root\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg6mx\" (UniqueName: \"kubernetes.io/projected/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-kube-api-access-dg6mx\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-wtmp\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-textfile\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-metrics-client-ca\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-sys\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:09.954846 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:09.954497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.054953 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.054912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-textfile\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.054968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-metrics-client-ca\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.054995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-sys\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-root\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg6mx\" (UniqueName: \"kubernetes.io/projected/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-kube-api-access-dg6mx\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-wtmp\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-textfile\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055482 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:04:10.055391    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:04:10.055482 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:04:10.055451    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls podName:fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a nodeName:}" failed. No retries permitted until 2026-07-28 19:04:10.55543591 +0000 UTC m=+117.997356399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls") pod "node-exporter-c2p2v" (UID: "fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a") : secret "node-exporter-tls" not found
Jul 28 19:04:10.055730 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-metrics-client-ca\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055730 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-root\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055829 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-sys\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.055941 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.055876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-wtmp\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.056159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.056136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.065138 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.065114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.068716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.068671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg6mx\" (UniqueName: \"kubernetes.io/projected/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-kube-api-access-dg6mx\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.559895 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.559853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.562706 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.562680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a-node-exporter-tls\") pod \"node-exporter-c2p2v\" (UID: \"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a\") " pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.763598 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.763562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2p2v"
Jul 28 19:04:10.776707 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:10.776680    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca36f13_08a2_4ccf_8ca4_7b8bed4ab39a.slice/crio-e78207bcdc11aa00a81b28f301594fed07902cc0c35069b1166db46fc2dfbdd2 WatchSource:0}: Error finding container e78207bcdc11aa00a81b28f301594fed07902cc0c35069b1166db46fc2dfbdd2: Status 404 returned error can't find the container with id e78207bcdc11aa00a81b28f301594fed07902cc0c35069b1166db46fc2dfbdd2
Jul 28 19:04:10.972043 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.971994    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:04:10.977362 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.977335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:10.980827 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.980803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981407    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981447    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981654    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981663    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981907    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ljtq\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.981908    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 19:04:10.982484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.982158    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 19:04:10.997890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:10.997863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:04:11.164206 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164602 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164602 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164602 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164602 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164872 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164872 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.164872 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.165018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxwp\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.165018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.164942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.265798 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.265704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.265798 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.265756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.265798 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.265783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.265913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.265991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxwp\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.266447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.266322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.268274 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.267359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.268274 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.267448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.270475 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.269493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.271533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.271486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.271809 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.271765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.272231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.272185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.272558 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.272536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.273098 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.273076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.273197 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.273177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.273291 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.273267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.273943 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.273906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.277699 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.277677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxwp\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp\") pod \"alertmanager-main-0\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.292424 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.292380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:04:11.468078 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.468046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:04:11.472324 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:11.472296    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2759e387_17dc_49e5_b6a2_138b61dcc6b2.slice/crio-32fe0e635fc0ba080b557f8ee13dded08790fb3dfb0ab2176335ba71eaa6c311 WatchSource:0}: Error finding container 32fe0e635fc0ba080b557f8ee13dded08790fb3dfb0ab2176335ba71eaa6c311: Status 404 returned error can't find the container with id 32fe0e635fc0ba080b557f8ee13dded08790fb3dfb0ab2176335ba71eaa6c311
Jul 28 19:04:11.559622 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.559527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"32fe0e635fc0ba080b557f8ee13dded08790fb3dfb0ab2176335ba71eaa6c311"}
Jul 28 19:04:11.560955 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.560884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p2v" event={"ID":"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a","Type":"ContainerStarted","Data":"e78207bcdc11aa00a81b28f301594fed07902cc0c35069b1166db46fc2dfbdd2"}
Jul 28 19:04:11.836911 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.836878    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"]
Jul 28 19:04:11.843740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.842100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.844829    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846373    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-410om9opjv9nc\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846817    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8jqvm\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846966    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 19:04:11.847083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.846970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 19:04:11.855944 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.855921    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"]
Jul 28 19:04:11.872977 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.872955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-grpc-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873094 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873143 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ffa8b2b-b9c8-445d-be7e-25448a27264a-metrics-client-ca\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873285 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873285 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.873375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.873300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6mm7\" (UniqueName: \"kubernetes.io/projected/2ffa8b2b-b9c8-445d-be7e-25448a27264a-kube-api-access-t6mm7\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ffa8b2b-b9c8-445d-be7e-25448a27264a-metrics-client-ca\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6mm7\" (UniqueName: \"kubernetes.io/projected/2ffa8b2b-b9c8-445d-be7e-25448a27264a-kube-api-access-t6mm7\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-grpc-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.974875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.974874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.975727 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.975692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ffa8b2b-b9c8-445d-be7e-25448a27264a-metrics-client-ca\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.978315 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.977892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.978315 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.978002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.978315 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.978296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.978506 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.978295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-grpc-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.978682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.978663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-tls\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.979105 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.979084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ffa8b2b-b9c8-445d-be7e-25448a27264a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:11.985095 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:11.985074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6mm7\" (UniqueName: \"kubernetes.io/projected/2ffa8b2b-b9c8-445d-be7e-25448a27264a-kube-api-access-t6mm7\") pod \"thanos-querier-7b55c6476b-z8vqd\" (UID: \"2ffa8b2b-b9c8-445d-be7e-25448a27264a\") " pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:12.154285 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:12.154182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:12.478318 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:12.478244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:04:13.775229 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:13.775170    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ffa8b2b_b9c8_445d_be7e_25448a27264a.slice/crio-dae0b36caef83d9495e9f0f8010bf6c123a1fe6e8937a372d5cf3de94f47b0ef WatchSource:0}: Error finding container dae0b36caef83d9495e9f0f8010bf6c123a1fe6e8937a372d5cf3de94f47b0ef: Status 404 returned error can't find the container with id dae0b36caef83d9495e9f0f8010bf6c123a1fe6e8937a372d5cf3de94f47b0ef
Jul 28 19:04:13.775736 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:13.775460    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"]
Jul 28 19:04:14.573277 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.573178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"dae0b36caef83d9495e9f0f8010bf6c123a1fe6e8937a372d5cf3de94f47b0ef"}
Jul 28 19:04:14.575186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.575157    2577 generic.go:356] "Generic (PLEG): container finished" podID="fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a" containerID="5da58dd243c71a08b00691e4fa5da13b8ec64446fce25a1a73077cb37803294b" exitCode=0
Jul 28 19:04:14.575339 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.575292    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p2v" event={"ID":"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a","Type":"ContainerDied","Data":"5da58dd243c71a08b00691e4fa5da13b8ec64446fce25a1a73077cb37803294b"}
Jul 28 19:04:14.577931 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.577901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jjvj7" event={"ID":"a075b9da-2afb-47fa-ac98-f30aac172c3e","Type":"ContainerStarted","Data":"2b340aee47c1a81de43cc6fe5715dc2dec18485ad90733d47de1c5d7fc97acdf"}
Jul 28 19:04:14.580588 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.580455    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995" exitCode=0
Jul 28 19:04:14.580588 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.580509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"}
Jul 28 19:04:14.679638 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:14.679482    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jjvj7" podStartSLOduration=33.91091372 podStartE2EDuration="46.679461607s" totalImagesPullingTime="12.768547887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:03:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:14.677269738 +0000 UTC m=+122.119190248" watchObservedRunningTime="2026-07-28 19:04:14.679461607 +0000 UTC m=+122.121382116"
Jul 28 19:04:15.052733 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.052701    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5787969df6-s9p6z"]
Jul 28 19:04:15.056570 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.056545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.060046 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.060017    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-btwd7\""
Jul 28 19:04:15.060656 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.060279    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 19:04:15.060656 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.060440    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 19:04:15.060656 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.060530    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 19:04:15.063768 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.063746    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 19:04:15.064019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.063994    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 19:04:15.070418 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.070396    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 19:04:15.076853 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.076315    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5787969df6-s9p6z"]
Jul 28 19:04:15.103859 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.103835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-metrics-client-ca\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.103994 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.103877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.103994 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.103913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.104106 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.104028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.104106 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.104089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-federate-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.104194 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.104110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrcx\" (UniqueName: \"kubernetes.io/projected/ef112191-715e-4213-8f8f-45f7f27253ce-kube-api-access-hjrcx\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.104194 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.104147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.104194 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.104168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-serving-certs-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-federate-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrcx\" (UniqueName: \"kubernetes.io/projected/ef112191-715e-4213-8f8f-45f7f27253ce-kube-api-access-hjrcx\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-serving-certs-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-metrics-client-ca\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.205890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.205766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.213033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.207246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.213033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.207667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-metrics-client-ca\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.213033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.207685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef112191-715e-4213-8f8f-45f7f27253ce-serving-certs-ca-bundle\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.213033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.208582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.213033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.210457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-telemeter-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.216375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.215140    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.216375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.215929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrcx\" (UniqueName: \"kubernetes.io/projected/ef112191-715e-4213-8f8f-45f7f27253ce-kube-api-access-hjrcx\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.216733 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.216709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef112191-715e-4213-8f8f-45f7f27253ce-federate-client-tls\") pod \"telemeter-client-5787969df6-s9p6z\" (UID: \"ef112191-715e-4213-8f8f-45f7f27253ce\") " pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.370121 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.370075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z"
Jul 28 19:04:15.587191 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.587149    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p2v" event={"ID":"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a","Type":"ContainerStarted","Data":"6a8f5c755d804304e2d53a99cfba9509897e4d71886c0f951089ac67ef12e3b6"}
Jul 28 19:04:15.587379 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.587264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p2v" event={"ID":"fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a","Type":"ContainerStarted","Data":"34b3642ada6b7039cd86d6aab5ac702bb9d70f21345e554b979f80a640ab712a"}
Jul 28 19:04:15.607345 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:15.607300    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c2p2v" podStartSLOduration=3.709031115 podStartE2EDuration="6.607285295s" totalImagesPullingTime="2.821432917s" totalInitContainerRuntime="76.821263ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:15.605667805 +0000 UTC m=+123.047588329" watchObservedRunningTime="2026-07-28 19:04:15.607285295 +0000 UTC m=+123.049205803"
Jul 28 19:04:16.385643 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:16.385582    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef112191_715e_4213_8f8f_45f7f27253ce.slice/crio-d3060ce691351fa4ed9143d53bab4dc6db54c73956e2e786429e6ef08429bef6 WatchSource:0}: Error finding container d3060ce691351fa4ed9143d53bab4dc6db54c73956e2e786429e6ef08429bef6: Status 404 returned error can't find the container with id d3060ce691351fa4ed9143d53bab4dc6db54c73956e2e786429e6ef08429bef6
Jul 28 19:04:16.393891 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.393852    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5787969df6-s9p6z"]
Jul 28 19:04:16.594823 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.594762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"}
Jul 28 19:04:16.594958 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.594844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"}
Jul 28 19:04:16.594958 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.594888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"}
Jul 28 19:04:16.594958 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.594938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"}
Jul 28 19:04:16.595118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.594978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"}
Jul 28 19:04:16.597996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.597943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"bb3788cd9ba9025c2c43dfdb5f4c152281585a276f6bb44a5e8cdd1a8cb577bf"}
Jul 28 19:04:16.598114 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.598017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"bde5102c48b9140b391bdd4085841be2a3530e6e7929fc6c3864d1242dca8daf"}
Jul 28 19:04:16.598114 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.598060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"c5317d773480bde0683a25cc59a361590a061dd935e4a4e004adfeae1ff8fbb1"}
Jul 28 19:04:16.599445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:16.599422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z" event={"ID":"ef112191-715e-4213-8f8f-45f7f27253ce","Type":"ContainerStarted","Data":"d3060ce691351fa4ed9143d53bab4dc6db54c73956e2e786429e6ef08429bef6"}
Jul 28 19:04:17.137244 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:17.136884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:04:17.608185 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:17.608145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerStarted","Data":"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"}
Jul 28 19:04:17.611390 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:17.611356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"9e4738e8a129db1fb9cd1d7f2fcc1f9d3dd455e3d8fe3424aebd157219d41ccf"}
Jul 28 19:04:17.611514 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:17.611419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"bdc6c8ea6dc61232e24a20b80e6ea6aa85769180f934b5fe58c5a55c4e7df8f4"}
Jul 28 19:04:17.641569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:17.641516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.438716698 podStartE2EDuration="7.641495846s" totalImagesPullingTime="5.184154986s" totalInitContainerRuntime="18.624162ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:17.639827316 +0000 UTC m=+125.081747825" watchObservedRunningTime="2026-07-28 19:04:17.641495846 +0000 UTC m=+125.083416355"
Jul 28 19:04:18.616856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.616813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z" event={"ID":"ef112191-715e-4213-8f8f-45f7f27253ce","Type":"ContainerStarted","Data":"2b9310fd45de292073c410a8eb3b44eb67ba40661001e954ca4da8a530fd9f6e"}
Jul 28 19:04:18.617319 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.616882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z" event={"ID":"ef112191-715e-4213-8f8f-45f7f27253ce","Type":"ContainerStarted","Data":"fbabc581f345241129f96c7f4f6b91db753f2e5586f669fc72f8b37382247042"}
Jul 28 19:04:18.617319 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.616903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z" event={"ID":"ef112191-715e-4213-8f8f-45f7f27253ce","Type":"ContainerStarted","Data":"7e27a00beb55bc0001af9dc11b26026670479c5f5f866d6b9bdb0230f2718b69"}
Jul 28 19:04:18.619675 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.619652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" event={"ID":"2ffa8b2b-b9c8-445d-be7e-25448a27264a","Type":"ContainerStarted","Data":"658e33851dce84868d9f398ef615ed0c9135a6bfbc2757b1796a07c960774dba"}
Jul 28 19:04:18.619773 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.619758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:18.640878 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.640827    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5787969df6-s9p6z" podStartSLOduration=2.029929524 podStartE2EDuration="3.640812453s" totalImagesPullingTime="1.610882929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:18.639436249 +0000 UTC m=+126.081356758" watchObservedRunningTime="2026-07-28 19:04:18.640812453 +0000 UTC m=+126.082732961"
Jul 28 19:04:18.670072 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:18.670021    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd" podStartSLOduration=4.354470129 podStartE2EDuration="7.670002045s" totalImagesPullingTime="3.315531916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:18.668552622 +0000 UTC m=+126.110473142" watchObservedRunningTime="2026-07-28 19:04:18.670002045 +0000 UTC m=+126.111922554"
Jul 28 19:04:22.770493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:22.770456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:04:22.772849 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:22.772825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea2fbe2e-e91e-416c-89d5-2f621a27990b-metrics-certs\") pod \"network-metrics-daemon-5g9jv\" (UID: \"ea2fbe2e-e91e-416c-89d5-2f621a27990b\") " pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:04:23.059765 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:23.059688    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:04:23.068482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:23.068467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5g9jv"
Jul 28 19:04:23.196869 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:23.196844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5g9jv"]
Jul 28 19:04:23.198655 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:04:23.198626    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2fbe2e_e91e_416c_89d5_2f621a27990b.slice/crio-93e36b4ea957240097c8fcc761a108f437d82eccfa50fa8e50ef9f40e37534f0 WatchSource:0}: Error finding container 93e36b4ea957240097c8fcc761a108f437d82eccfa50fa8e50ef9f40e37534f0: Status 404 returned error can't find the container with id 93e36b4ea957240097c8fcc761a108f437d82eccfa50fa8e50ef9f40e37534f0
Jul 28 19:04:23.640010 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:23.639969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5g9jv" event={"ID":"ea2fbe2e-e91e-416c-89d5-2f621a27990b","Type":"ContainerStarted","Data":"93e36b4ea957240097c8fcc761a108f437d82eccfa50fa8e50ef9f40e37534f0"}
Jul 28 19:04:24.632258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:24.632230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b55c6476b-z8vqd"
Jul 28 19:04:25.649406 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:25.649370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5g9jv" event={"ID":"ea2fbe2e-e91e-416c-89d5-2f621a27990b","Type":"ContainerStarted","Data":"0af87a17b5c9cc03fadc7a28ee8c41e10b5b311f3d3279570a1ebbc57cff50ae"}
Jul 28 19:04:25.649784 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:25.649421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5g9jv" event={"ID":"ea2fbe2e-e91e-416c-89d5-2f621a27990b","Type":"ContainerStarted","Data":"9ca01175245e52276618c31a85a1320532979c0baca06cb55d58fc4d70884486"}
Jul 28 19:04:25.665884 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:25.665835    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5g9jv" podStartSLOduration=130.946849888 podStartE2EDuration="2m12.665821091s" totalImagesPullingTime="1.718971203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:25.665392695 +0000 UTC m=+133.107313202" watchObservedRunningTime="2026-07-28 19:04:25.665821091 +0000 UTC m=+133.107741598"
Jul 28 19:04:42.161004 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.160965    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerName="registry" containerID="cri-o://bca4c8f722cef9c3c090d104904ac69af8d9d6d21f56bd251ccd8945230d0ac1" gracePeriod=30
Jul 28 19:04:42.474023 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.473945    2577 patch_prober.go:28] interesting pod/image-registry-db948c5b7-cpw58 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body=
Jul 28 19:04:42.474023 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.473998    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused"
Jul 28 19:04:42.718509 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.718478    2577 generic.go:356] "Generic (PLEG): container finished" podID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerID="bca4c8f722cef9c3c090d104904ac69af8d9d6d21f56bd251ccd8945230d0ac1" exitCode=0
Jul 28 19:04:42.718639 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.718552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" event={"ID":"7c2ba24f-5a21-4010-8024-3f363c23a8a5","Type":"ContainerDied","Data":"bca4c8f722cef9c3c090d104904ac69af8d9d6d21f56bd251ccd8945230d0ac1"}
Jul 28 19:04:42.719857 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.719835    2577 generic.go:356] "Generic (PLEG): container finished" podID="77642228-1e12-416d-9ddf-a8168eb4823e" containerID="ee0da683bd3aa426e163c2209e96d53074212173a3838a04b9aeee693dcd635d" exitCode=0
Jul 28 19:04:42.719949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.719865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh" event={"ID":"77642228-1e12-416d-9ddf-a8168eb4823e","Type":"ContainerDied","Data":"ee0da683bd3aa426e163c2209e96d53074212173a3838a04b9aeee693dcd635d"}
Jul 28 19:04:42.720132 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.720120    2577 scope.go:117] "RemoveContainer" containerID="ee0da683bd3aa426e163c2209e96d53074212173a3838a04b9aeee693dcd635d"
Jul 28 19:04:42.947422 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:42.947397    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:04:43.025996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.025931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.025996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.025969    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.025996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.025993    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.026188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026030    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zthk\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.026188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.026188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.026188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.026417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates\") pod \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\" (UID: \"7c2ba24f-5a21-4010-8024-3f363c23a8a5\") "
Jul 28 19:04:43.027040 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:04:43.027040 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.026926    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:04:43.028739 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.028695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:04:43.028839 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.028774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:04:43.028885 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.028839    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:43.028885 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.028850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:43.029105 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.029088    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk" (OuterVolumeSpecName: "kube-api-access-2zthk") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "kube-api-access-2zthk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:43.036898 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.036873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7c2ba24f-5a21-4010-8024-3f363c23a8a5" (UID: "7c2ba24f-5a21-4010-8024-3f363c23a8a5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:04:43.127108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127085    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-bound-sa-token\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127107    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-image-registry-private-configuration\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127123    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-certificates\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127133    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2ba24f-5a21-4010-8024-3f363c23a8a5-trusted-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127142    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c2ba24f-5a21-4010-8024-3f363c23a8a5-installation-pull-secrets\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127150    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-registry-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127158    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zthk\" (UniqueName: \"kubernetes.io/projected/7c2ba24f-5a21-4010-8024-3f363c23a8a5-kube-api-access-2zthk\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.127245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.127165    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c2ba24f-5a21-4010-8024-3f363c23a8a5-ca-trust-extracted\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:04:43.725016 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.724975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db948c5b7-cpw58" event={"ID":"7c2ba24f-5a21-4010-8024-3f363c23a8a5","Type":"ContainerDied","Data":"7e9de5dc85026050b1b8295fbb3c4a594d4983c7933c19225477f5be5307b1cb"}
Jul 28 19:04:43.725488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.725071    2577 scope.go:117] "RemoveContainer" containerID="bca4c8f722cef9c3c090d104904ac69af8d9d6d21f56bd251ccd8945230d0ac1"
Jul 28 19:04:43.725488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.724989    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db948c5b7-cpw58"
Jul 28 19:04:43.727089 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.727062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh" event={"ID":"77642228-1e12-416d-9ddf-a8168eb4823e","Type":"ContainerStarted","Data":"e34317232e0968ec74015ccd7025825fa2418a4950afefb57d4296cc82e97fd1"}
Jul 28 19:04:43.760888 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.760860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:04:43.765431 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:43.765408    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-db948c5b7-cpw58"]
Jul 28 19:04:45.050856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:45.050825    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" path="/var/lib/kubelet/pods/7c2ba24f-5a21-4010-8024-3f363c23a8a5/volumes"
Jul 28 19:04:49.743682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:04:49.743657    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9trb9_560782e8-1ff7-4fa5-b66c-fe5ee1e1f176/serve-healthcheck-canary/0.log"
Jul 28 19:05:30.262007 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.261975    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:30.262559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.262515    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="alertmanager" containerID="cri-o://c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49" gracePeriod=120
Jul 28 19:05:30.262636 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.262563    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-web" containerID="cri-o://09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21" gracePeriod=120
Jul 28 19:05:30.262713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.262685    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="config-reloader" containerID="cri-o://a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14" gracePeriod=120
Jul 28 19:05:30.262774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.262703    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy" containerID="cri-o://3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b" gracePeriod=120
Jul 28 19:05:30.263404 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.262963    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="prom-label-proxy" containerID="cri-o://2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b" gracePeriod=120
Jul 28 19:05:30.263404 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.263071    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-metric" containerID="cri-o://05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940" gracePeriod=120
Jul 28 19:05:30.919803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919762    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b" exitCode=0
Jul 28 19:05:30.919803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919794    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b" exitCode=0
Jul 28 19:05:30.919803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919801    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14" exitCode=0
Jul 28 19:05:30.919803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919809    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49" exitCode=0
Jul 28 19:05:30.920065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"}
Jul 28 19:05:30.920065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"}
Jul 28 19:05:30.920065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"}
Jul 28 19:05:30.920065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:30.919956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"}
Jul 28 19:05:31.601610 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.601589    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:31.693165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693179    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693255    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693316    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693346    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbxwp\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693372    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693440    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693465    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693494    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693535    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693568    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy\") pod \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\" (UID: \"2759e387-17dc-49e5-b6a2-138b61dcc6b2\") "
Jul 28 19:05:31.693915 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693593    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:31.693915 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.693791    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.694050 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.694023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:31.694461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.694375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:05:31.696188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.696121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:05:31.696581 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.696544    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.696837 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.696774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out" (OuterVolumeSpecName: "config-out") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:05:31.696837 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.696781    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.696837 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.696827    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp" (OuterVolumeSpecName: "kube-api-access-rbxwp") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "kube-api-access-rbxwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:05:31.697287 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.697263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.697447 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.697414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume" (OuterVolumeSpecName: "config-volume") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.698145 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.698128    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.700265 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.700239    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.706714 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.706695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config" (OuterVolumeSpecName: "web-config") pod "2759e387-17dc-49e5-b6a2-138b61dcc6b2" (UID: "2759e387-17dc-49e5-b6a2-138b61dcc6b2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:31.794192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794162    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-main-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794194    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794205    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-tls-assets\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794242    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2759e387-17dc-49e5-b6a2-138b61dcc6b2-metrics-client-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794256    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-cluster-tls-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794270    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794281    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-out\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794288    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbxwp\" (UniqueName: \"kubernetes.io/projected/2759e387-17dc-49e5-b6a2-138b61dcc6b2-kube-api-access-rbxwp\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794296    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2759e387-17dc-49e5-b6a2-138b61dcc6b2-alertmanager-main-db\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794304    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-config-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794312    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-web-config\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.794324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.794321    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2759e387-17dc-49e5-b6a2-138b61dcc6b2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:05:31.926188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926119    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940" exitCode=0
Jul 28 19:05:31.926188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926142    2577 generic.go:356] "Generic (PLEG): container finished" podID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerID="09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21" exitCode=0
Jul 28 19:05:31.926372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"}
Jul 28 19:05:31.926372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926230    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:31.926372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"}
Jul 28 19:05:31.926372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2759e387-17dc-49e5-b6a2-138b61dcc6b2","Type":"ContainerDied","Data":"32fe0e635fc0ba080b557f8ee13dded08790fb3dfb0ab2176335ba71eaa6c311"}
Jul 28 19:05:31.926372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.926354    2577 scope.go:117] "RemoveContainer" containerID="2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"
Jul 28 19:05:31.934247 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.934209    2577 scope.go:117] "RemoveContainer" containerID="05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"
Jul 28 19:05:31.941037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.941018    2577 scope.go:117] "RemoveContainer" containerID="3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"
Jul 28 19:05:31.947935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.947918    2577 scope.go:117] "RemoveContainer" containerID="09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"
Jul 28 19:05:31.949046 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.949029    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:31.952111 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.952092    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:31.956136 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.955661    2577 scope.go:117] "RemoveContainer" containerID="a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"
Jul 28 19:05:31.962635 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.962621    2577 scope.go:117] "RemoveContainer" containerID="c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"
Jul 28 19:05:31.969065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.969050    2577 scope.go:117] "RemoveContainer" containerID="a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"
Jul 28 19:05:31.975535 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.975490    2577 scope.go:117] "RemoveContainer" containerID="2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"
Jul 28 19:05:31.975998 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.975969    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b\": container with ID starting with 2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b not found: ID does not exist" containerID="2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"
Jul 28 19:05:31.976122 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976009    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"} err="failed to get container status \"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b\": rpc error: code = NotFound desc = could not find container \"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b\": container with ID starting with 2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b not found: ID does not exist"
Jul 28 19:05:31.976122 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976048    2577 scope.go:117] "RemoveContainer" containerID="05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"
Jul 28 19:05:31.976397 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.976375    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940\": container with ID starting with 05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940 not found: ID does not exist" containerID="05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"
Jul 28 19:05:31.976490 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976403    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"} err="failed to get container status \"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940\": rpc error: code = NotFound desc = could not find container \"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940\": container with ID starting with 05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940 not found: ID does not exist"
Jul 28 19:05:31.976490 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976421    2577 scope.go:117] "RemoveContainer" containerID="3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"
Jul 28 19:05:31.976708 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.976688    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b\": container with ID starting with 3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b not found: ID does not exist" containerID="3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"
Jul 28 19:05:31.976753 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976716    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"} err="failed to get container status \"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b\": rpc error: code = NotFound desc = could not find container \"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b\": container with ID starting with 3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b not found: ID does not exist"
Jul 28 19:05:31.976753 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976750    2577 scope.go:117] "RemoveContainer" containerID="09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"
Jul 28 19:05:31.976984 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.976962    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21\": container with ID starting with 09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21 not found: ID does not exist" containerID="09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"
Jul 28 19:05:31.977056 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.976992    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"} err="failed to get container status \"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21\": rpc error: code = NotFound desc = could not find container \"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21\": container with ID starting with 09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21 not found: ID does not exist"
Jul 28 19:05:31.977056 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977014    2577 scope.go:117] "RemoveContainer" containerID="a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"
Jul 28 19:05:31.977293 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.977276    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14\": container with ID starting with a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14 not found: ID does not exist" containerID="a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"
Jul 28 19:05:31.977341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977296    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"} err="failed to get container status \"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14\": rpc error: code = NotFound desc = could not find container \"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14\": container with ID starting with a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14 not found: ID does not exist"
Jul 28 19:05:31.977341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977309    2577 scope.go:117] "RemoveContainer" containerID="c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"
Jul 28 19:05:31.977341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977326    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:31.977535 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.977517    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49\": container with ID starting with c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49 not found: ID does not exist" containerID="c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"
Jul 28 19:05:31.977599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977541    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"} err="failed to get container status \"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49\": rpc error: code = NotFound desc = could not find container \"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49\": container with ID starting with c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49 not found: ID does not exist"
Jul 28 19:05:31.977599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977560    2577 scope.go:117] "RemoveContainer" containerID="a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"
Jul 28 19:05:31.977599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977581    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="prom-label-proxy"
Jul 28 19:05:31.977599 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977596    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="prom-label-proxy"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977607    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977613    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977630    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-metric"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977639    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-metric"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="init-config-reloader"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977665    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="init-config-reloader"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977677    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="alertmanager"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977685    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="alertmanager"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977696    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-web"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977703    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-web"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977717    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerName="registry"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977725    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerName="registry"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977734    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="config-reloader"
Jul 28 19:05:31.977748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977740    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="config-reloader"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977791    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="alertmanager"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977799    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="prom-label-proxy"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977812    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-web"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977818    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy-metric"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977824    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="kube-rbac-proxy"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977829    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" containerName="config-reloader"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977834    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c2ba24f-5a21-4010-8024-3f363c23a8a5" containerName="registry"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:05:31.977795    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995\": container with ID starting with a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995 not found: ID does not exist" containerID="a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977884    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"} err="failed to get container status \"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995\": rpc error: code = NotFound desc = could not find container \"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995\": container with ID starting with a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995 not found: ID does not exist"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.977898    2577 scope.go:117] "RemoveContainer" containerID="2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978103    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b"} err="failed to get container status \"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b\": rpc error: code = NotFound desc = could not find container \"2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b\": container with ID starting with 2a816d83c6114740e0836c199ba65edc70914e880d547648979210f9b19a9a2b not found: ID does not exist"
Jul 28 19:05:31.978238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978123    2577 scope.go:117] "RemoveContainer" containerID="05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"
Jul 28 19:05:31.978643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978336    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940"} err="failed to get container status \"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940\": rpc error: code = NotFound desc = could not find container \"05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940\": container with ID starting with 05fb4f1390cc89e9cb735e7365918310e2ed248caf019b29bd39ab733eed0940 not found: ID does not exist"
Jul 28 19:05:31.978643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978353    2577 scope.go:117] "RemoveContainer" containerID="3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"
Jul 28 19:05:31.978643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978542    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b"} err="failed to get container status \"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b\": rpc error: code = NotFound desc = could not find container \"3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b\": container with ID starting with 3652ac1774bb72ac3b58a8402b751ffe88d276a1703b46862ab0353462b17f4b not found: ID does not exist"
Jul 28 19:05:31.978643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978558    2577 scope.go:117] "RemoveContainer" containerID="09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"
Jul 28 19:05:31.978787 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978740    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21"} err="failed to get container status \"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21\": rpc error: code = NotFound desc = could not find container \"09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21\": container with ID starting with 09861a6730ce3de7962acd06abf0994705b32c631b5a3f52fd5f6a44f95d3f21 not found: ID does not exist"
Jul 28 19:05:31.978787 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978753    2577 scope.go:117] "RemoveContainer" containerID="a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"
Jul 28 19:05:31.978952 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978934    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14"} err="failed to get container status \"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14\": rpc error: code = NotFound desc = could not find container \"a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14\": container with ID starting with a70e84274aac1232ce4155489d3895adaa566abf794d57a74bd769817025ce14 not found: ID does not exist"
Jul 28 19:05:31.979019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.978955    2577 scope.go:117] "RemoveContainer" containerID="c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"
Jul 28 19:05:31.979169 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.979152    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49"} err="failed to get container status \"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49\": rpc error: code = NotFound desc = could not find container \"c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49\": container with ID starting with c7f0a51b6ccf457d4da5dc4b9bd85b618f6ab6ffc33db7645715014c2058da49 not found: ID does not exist"
Jul 28 19:05:31.979249 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.979169    2577 scope.go:117] "RemoveContainer" containerID="a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"
Jul 28 19:05:31.979408 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.979391    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995"} err="failed to get container status \"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995\": rpc error: code = NotFound desc = could not find container \"a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995\": container with ID starting with a05a2b7dbb2e4cce3657cd9d850d79d673f188f19ececb8a96ca74389b2ea995 not found: ID does not exist"
Jul 28 19:05:31.982838 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.982820    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:31.985207 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ljtq\""
Jul 28 19:05:31.985207 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 19:05:31.985425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 19:05:31.985425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985276    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 19:05:31.985425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 19:05:31.985425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985334    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 19:05:31.985425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985369    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 19:05:31.985691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985509    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 19:05:31.985752 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.985706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 19:05:31.990587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.990568    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 19:05:31.994647 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:31.994624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:32.095080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095201 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095294 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxs5x\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-kube-api-access-sxs5x\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095294 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095450 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-out\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095450 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095529 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095529 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095529 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-web-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-volume\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.095626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.095587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.195961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.195891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxs5x\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-kube-api-access-sxs5x\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.195961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.195931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.195961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.195956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.195981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-out\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-web-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-volume\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.196511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.196501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.197463 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.197433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.197811 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.197790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.198866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.198822    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-out\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.198961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.198913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.199146 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.199120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-web-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.199440 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.199410    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.199440 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.199410    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.199688 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.199665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.199688 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.199681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.200073 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.200050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-config-volume\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.200646 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.200626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.203426 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.203409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxs5x\" (UniqueName: \"kubernetes.io/projected/d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650-kube-api-access-sxs5x\") pod \"alertmanager-main-0\" (UID: \"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.291898 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.291876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:05:32.426660 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.426518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:05:32.430795 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:05:32.430758    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f50fa0_d0d7_4db8_bd2c_20e30c7d0650.slice/crio-012859edbd2ee3424134f0cc2428ca1a173af57b5d3ba8d79c50810ec9ccb8db WatchSource:0}: Error finding container 012859edbd2ee3424134f0cc2428ca1a173af57b5d3ba8d79c50810ec9ccb8db: Status 404 returned error can't find the container with id 012859edbd2ee3424134f0cc2428ca1a173af57b5d3ba8d79c50810ec9ccb8db
Jul 28 19:05:32.932230 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.932189    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650" containerID="b44a1c580ec1eb649c59b8d4d6a49b46af765e18575d20444d1b9dcad0f38a85" exitCode=0
Jul 28 19:05:32.932582 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.932242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerDied","Data":"b44a1c580ec1eb649c59b8d4d6a49b46af765e18575d20444d1b9dcad0f38a85"}
Jul 28 19:05:32.932582 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:32.932303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"012859edbd2ee3424134f0cc2428ca1a173af57b5d3ba8d79c50810ec9ccb8db"}
Jul 28 19:05:33.050701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.050675    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2759e387-17dc-49e5-b6a2-138b61dcc6b2" path="/var/lib/kubelet/pods/2759e387-17dc-49e5-b6a2-138b61dcc6b2/volumes"
Jul 28 19:05:33.939764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"1e3387e84a29462a395fcd3120dd73ed4658e2b611f63d8a839d0afebe8d6263"}
Jul 28 19:05:33.940234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"5d56e06d35c3113aaf34f8e0956c65a23827ea48ae146220841c9fb6a47db21e"}
Jul 28 19:05:33.940234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939833    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"53faaa6c4a612b3b7a387d14c7961a60ab7b84c465d72944455d3d1737e2d7f2"}
Jul 28 19:05:33.940234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"bb6cd319919faef3886126800fead91d55e4907581d913517cb6945bdff63578"}
Jul 28 19:05:33.940234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"f38d238a5654ffce62f697400fa284f680587f2f2fe4a0fc68b7a3cde27cc622"}
Jul 28 19:05:33.940234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.939903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650","Type":"ContainerStarted","Data":"6e1eea653e53a2732a8e3f47ec51ff9b0c26ebf0f294076645e2f332bb1bfac8"}
Jul 28 19:05:33.967067 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:05:33.967019    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.952043534 podStartE2EDuration="2.967004317s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.960783ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:05:33.96534966 +0000 UTC m=+201.407270167" watchObservedRunningTime="2026-07-28 19:05:33.967004317 +0000 UTC m=+201.408924826"
Jul 28 19:06:15.388054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.388021    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"]
Jul 28 19:06:15.392501 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.392484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.395125 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.395102    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:06:15.395251 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.395104    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vxk74\""
Jul 28 19:06:15.396057 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.396037    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:06:15.400942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.400919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"]
Jul 28 19:06:15.507838 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.507812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.507942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.507839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grpq\" (UniqueName: \"kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.507942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.507867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.608374 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.608339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.608374 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.608376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grpq\" (UniqueName: \"kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.608487 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.608403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.608721 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.608701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.608763 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.608737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.616977 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.616958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grpq\" (UniqueName: \"kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.701967 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.701913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:15.825434 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:15.825413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"]
Jul 28 19:06:15.828027 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:15.828001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef7a168_daec_4004_aacd_0240740e37c5.slice/crio-43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707 WatchSource:0}: Error finding container 43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707: Status 404 returned error can't find the container with id 43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707
Jul 28 19:06:16.102714 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:16.102684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerStarted","Data":"43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707"}
Jul 28 19:06:24.133860 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:24.133824    2577 generic.go:356] "Generic (PLEG): container finished" podID="eef7a168-daec-4004-aacd-0240740e37c5" containerID="6408e4bec7233630b0b15ed135c286a83efa609f06a499d39df025fdeba5bf26" exitCode=0
Jul 28 19:06:24.134372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:24.133868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerDied","Data":"6408e4bec7233630b0b15ed135c286a83efa609f06a499d39df025fdeba5bf26"}
Jul 28 19:06:26.143466 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:26.143438    2577 generic.go:356] "Generic (PLEG): container finished" podID="eef7a168-daec-4004-aacd-0240740e37c5" containerID="38c894953ccf99b73512360fd33350ab3abafca6b6664123cfb2431dac238a66" exitCode=0
Jul 28 19:06:26.143889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:26.143485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerDied","Data":"38c894953ccf99b73512360fd33350ab3abafca6b6664123cfb2431dac238a66"}
Jul 28 19:06:33.176785 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:33.176752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerStarted","Data":"58744e8bb53a480c2228bca25e37543a93ad7c281a6be89b452a38baffd1c0d8"}
Jul 28 19:06:34.182411 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.182380    2577 generic.go:356] "Generic (PLEG): container finished" podID="eef7a168-daec-4004-aacd-0240740e37c5" containerID="58744e8bb53a480c2228bca25e37543a93ad7c281a6be89b452a38baffd1c0d8" exitCode=0
Jul 28 19:06:34.182902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.182470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerDied","Data":"58744e8bb53a480c2228bca25e37543a93ad7c281a6be89b452a38baffd1c0d8"}
Jul 28 19:06:34.309911 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.309890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:34.359803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.359777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle\") pod \"eef7a168-daec-4004-aacd-0240740e37c5\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") "
Jul 28 19:06:34.359964 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.359817    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6grpq\" (UniqueName: \"kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq\") pod \"eef7a168-daec-4004-aacd-0240740e37c5\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") "
Jul 28 19:06:34.359964 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.359844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util\") pod \"eef7a168-daec-4004-aacd-0240740e37c5\" (UID: \"eef7a168-daec-4004-aacd-0240740e37c5\") "
Jul 28 19:06:34.360525 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.360503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle" (OuterVolumeSpecName: "bundle") pod "eef7a168-daec-4004-aacd-0240740e37c5" (UID: "eef7a168-daec-4004-aacd-0240740e37c5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:06:34.361879 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.361861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq" (OuterVolumeSpecName: "kube-api-access-6grpq") pod "eef7a168-daec-4004-aacd-0240740e37c5" (UID: "eef7a168-daec-4004-aacd-0240740e37c5"). InnerVolumeSpecName "kube-api-access-6grpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:06:34.365546 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.365519    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util" (OuterVolumeSpecName: "util") pod "eef7a168-daec-4004-aacd-0240740e37c5" (UID: "eef7a168-daec-4004-aacd-0240740e37c5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:06:34.461389 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.461314    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:34.461389 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.461340    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6grpq\" (UniqueName: \"kubernetes.io/projected/eef7a168-daec-4004-aacd-0240740e37c5-kube-api-access-6grpq\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:34.461389 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:34.461350    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eef7a168-daec-4004-aacd-0240740e37c5-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:35.188471 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:35.188404    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc"
Jul 28 19:06:35.188886 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:35.188406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404mdtc" event={"ID":"eef7a168-daec-4004-aacd-0240740e37c5","Type":"ContainerDied","Data":"43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707"}
Jul 28 19:06:35.188886 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:35.188520    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43e281d351b474ac64b25e79f0ee8d07f1031add4d8dd5af614779b5331d8707"
Jul 28 19:06:38.416127 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416092    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"]
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416397    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="util"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416408    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="util"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416430    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="pull"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416436    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="pull"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416442    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="extract"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416448    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="extract"
Jul 28 19:06:38.416519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.416515    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef7a168-daec-4004-aacd-0240740e37c5" containerName="extract"
Jul 28 19:06:38.419394 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.419380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.421905 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.421876    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 19:06:38.422126 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.422112    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8zf6w\""
Jul 28 19:06:38.422343 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.422330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:06:38.434164 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.434141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"]
Jul 28 19:06:38.489682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.489643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl7hg\" (UniqueName: \"kubernetes.io/projected/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-kube-api-access-pl7hg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.489809 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.489698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.590281 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.590249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.590407 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.590335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl7hg\" (UniqueName: \"kubernetes.io/projected/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-kube-api-access-pl7hg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.590605 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.590585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.598919 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.598894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl7hg\" (UniqueName: \"kubernetes.io/projected/5f10e4cf-c7d4-4015-b89b-9b7ad002b478-kube-api-access-pl7hg\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zx8g\" (UID: \"5f10e4cf-c7d4-4015-b89b-9b7ad002b478\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.728426 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.728349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"
Jul 28 19:06:38.858306 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:38.858273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g"]
Jul 28 19:06:38.862375 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:38.862344    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f10e4cf_c7d4_4015_b89b_9b7ad002b478.slice/crio-8a7ccc551a09483a1e6a3df308df4ba06d2eb1ed0e18321719b082ed8ead0651 WatchSource:0}: Error finding container 8a7ccc551a09483a1e6a3df308df4ba06d2eb1ed0e18321719b082ed8ead0651: Status 404 returned error can't find the container with id 8a7ccc551a09483a1e6a3df308df4ba06d2eb1ed0e18321719b082ed8ead0651
Jul 28 19:06:39.204566 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:39.204531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g" event={"ID":"5f10e4cf-c7d4-4015-b89b-9b7ad002b478","Type":"ContainerStarted","Data":"8a7ccc551a09483a1e6a3df308df4ba06d2eb1ed0e18321719b082ed8ead0651"}
Jul 28 19:06:42.217650 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:42.217611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g" event={"ID":"5f10e4cf-c7d4-4015-b89b-9b7ad002b478","Type":"ContainerStarted","Data":"85f61baa2634f9cbf8c2759a0f29ddb7fcb48b6381f35b1562753403b6c070a5"}
Jul 28 19:06:42.241517 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:42.241466    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zx8g" podStartSLOduration=1.7970305739999999 podStartE2EDuration="4.241448427s" totalImagesPullingTime="2.444417853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:42.23935038 +0000 UTC m=+269.681270900" watchObservedRunningTime="2026-07-28 19:06:42.241448427 +0000 UTC m=+269.683368938"
Jul 28 19:06:43.055281 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.055250    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"]
Jul 28 19:06:43.077198 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.077177    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"]
Jul 28 19:06:43.077361 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.077297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.080128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.079844    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:06:43.080128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.079916    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:06:43.080509 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.080486    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vxk74\""
Jul 28 19:06:43.131545 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.131519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.131665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.131564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.131665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.131605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7846j\" (UniqueName: \"kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.232494 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.232472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.232841 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.232511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.232841 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.232636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7846j\" (UniqueName: \"kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.232908 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.232853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.232908 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.232864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.241233 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.241190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7846j\" (UniqueName: \"kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.289239 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.289198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cc82l"]
Jul 28 19:06:43.311149 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.311090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cc82l"]
Jul 28 19:06:43.311255 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.311198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.313684 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.313656    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 19:06:43.313684 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.313670    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 19:06:43.313843 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.313699    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q55nh\""
Jul 28 19:06:43.333079 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.333050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.333175 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.333083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shk8t\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-kube-api-access-shk8t\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.386732 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.386711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:43.434094 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.434067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.434258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.434106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shk8t\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-kube-api-access-shk8t\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.450652 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.450603    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.450828 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.450808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shk8t\" (UniqueName: \"kubernetes.io/projected/bcb8734a-11c3-48d8-ac36-31d5612e362d-kube-api-access-shk8t\") pod \"cert-manager-webhook-6486ffd997-cc82l\" (UID: \"bcb8734a-11c3-48d8-ac36-31d5612e362d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.532875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.532844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"]
Jul 28 19:06:43.535638 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:43.535615    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e9cfc1_0945_4375_a27f_7e3322e20a0a.slice/crio-bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f WatchSource:0}: Error finding container bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f: Status 404 returned error can't find the container with id bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f
Jul 28 19:06:43.632179 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.632158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:43.778985 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:43.778959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cc82l"]
Jul 28 19:06:43.781387 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:43.781360    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcb8734a_11c3_48d8_ac36_31d5612e362d.slice/crio-d81407befa0a41b4087f312b951cbdbdb8aaf9eb32074f4bdd2e1e1f8449895c WatchSource:0}: Error finding container d81407befa0a41b4087f312b951cbdbdb8aaf9eb32074f4bdd2e1e1f8449895c: Status 404 returned error can't find the container with id d81407befa0a41b4087f312b951cbdbdb8aaf9eb32074f4bdd2e1e1f8449895c
Jul 28 19:06:44.227114 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.227082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l" event={"ID":"bcb8734a-11c3-48d8-ac36-31d5612e362d","Type":"ContainerStarted","Data":"d81407befa0a41b4087f312b951cbdbdb8aaf9eb32074f4bdd2e1e1f8449895c"}
Jul 28 19:06:44.228622 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.228594    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerID="6229098f6dae0265ec071de3d5e502b629a85e78b76e945037c6b82a3b41eea6" exitCode=0
Jul 28 19:06:44.228729 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.228644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt" event={"ID":"a3e9cfc1-0945-4375-a27f-7e3322e20a0a","Type":"ContainerDied","Data":"6229098f6dae0265ec071de3d5e502b629a85e78b76e945037c6b82a3b41eea6"}
Jul 28 19:06:44.228729 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.228680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt" event={"ID":"a3e9cfc1-0945-4375-a27f-7e3322e20a0a","Type":"ContainerStarted","Data":"bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f"}
Jul 28 19:06:44.876784 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.876750    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"]
Jul 28 19:06:44.892018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.891994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"]
Jul 28 19:06:44.892165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.892105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:44.894587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.894556    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jrft7\""
Jul 28 19:06:44.948643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.948617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:44.948784 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:44.948686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8km4f\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-kube-api-access-8km4f\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.049192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.049158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.049392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.049252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8km4f\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-kube-api-access-8km4f\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.057971 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.057916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.058106 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.058063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8km4f\" (UniqueName: \"kubernetes.io/projected/c4529292-4fef-4b60-95a5-6192cb9a3125-kube-api-access-8km4f\") pod \"cert-manager-cainjector-559cdd8c87-28mp4\" (UID: \"c4529292-4fef-4b60-95a5-6192cb9a3125\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.204243 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.204159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"
Jul 28 19:06:45.371044 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:45.371008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-28mp4"]
Jul 28 19:06:45.375194 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:45.375166    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4529292_4fef_4b60_95a5_6192cb9a3125.slice/crio-bc54af6b5b30391e376d3e7330cf9e2aa5a80066b4809115b8f8fe7fb7524baf WatchSource:0}: Error finding container bc54af6b5b30391e376d3e7330cf9e2aa5a80066b4809115b8f8fe7fb7524baf: Status 404 returned error can't find the container with id bc54af6b5b30391e376d3e7330cf9e2aa5a80066b4809115b8f8fe7fb7524baf
Jul 28 19:06:46.242268 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:46.242232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4" event={"ID":"c4529292-4fef-4b60-95a5-6192cb9a3125","Type":"ContainerStarted","Data":"bc54af6b5b30391e376d3e7330cf9e2aa5a80066b4809115b8f8fe7fb7524baf"}
Jul 28 19:06:48.253333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.253294    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerID="e144eb35d0947386c7e2be582bcdfc49a38a37d78c3221ba560f458973ad0a04" exitCode=0
Jul 28 19:06:48.253768 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.253341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt" event={"ID":"a3e9cfc1-0945-4375-a27f-7e3322e20a0a","Type":"ContainerDied","Data":"e144eb35d0947386c7e2be582bcdfc49a38a37d78c3221ba560f458973ad0a04"}
Jul 28 19:06:48.254919 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.254897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l" event={"ID":"bcb8734a-11c3-48d8-ac36-31d5612e362d","Type":"ContainerStarted","Data":"5386b7aba48dae90e57ce901b2db478f30c5a99f5a97b87ed95f507e20bad734"}
Jul 28 19:06:48.254997 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.254964    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:48.256411 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.256386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4" event={"ID":"c4529292-4fef-4b60-95a5-6192cb9a3125","Type":"ContainerStarted","Data":"27cdb0263c644e84f507a53ab747e5b1ff6f9f6f6b494045b8a7acbe48c5c735"}
Jul 28 19:06:48.309487 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.309439    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l" podStartSLOduration=1.305917157 podStartE2EDuration="5.309423266s" totalImagesPullingTime="4.003506109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:48.307712339 +0000 UTC m=+275.749632845" watchObservedRunningTime="2026-07-28 19:06:48.309423266 +0000 UTC m=+275.751343773"
Jul 28 19:06:48.334969 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:48.334928    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-28mp4" podStartSLOduration=1.606844819 podStartE2EDuration="4.334915183s" totalImagesPullingTime="2.728070364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:48.333780452 +0000 UTC m=+275.775700963" watchObservedRunningTime="2026-07-28 19:06:48.334915183 +0000 UTC m=+275.776835691"
Jul 28 19:06:49.263159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:49.263130    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerID="812bf3beca136d3972ebddba2ac7bda8a52e59f377601e9f097fc6ca21f4c4e6" exitCode=0
Jul 28 19:06:49.263585 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:49.263239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt" event={"ID":"a3e9cfc1-0945-4375-a27f-7e3322e20a0a","Type":"ContainerDied","Data":"812bf3beca136d3972ebddba2ac7bda8a52e59f377601e9f097fc6ca21f4c4e6"}
Jul 28 19:06:50.391011 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.390985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:50.496053 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.496024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7846j\" (UniqueName: \"kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j\") pod \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") "
Jul 28 19:06:50.496248 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.496061    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle\") pod \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") "
Jul 28 19:06:50.496248 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.496194    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util\") pod \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\" (UID: \"a3e9cfc1-0945-4375-a27f-7e3322e20a0a\") "
Jul 28 19:06:50.496520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.496499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle" (OuterVolumeSpecName: "bundle") pod "a3e9cfc1-0945-4375-a27f-7e3322e20a0a" (UID: "a3e9cfc1-0945-4375-a27f-7e3322e20a0a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:06:50.498129 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.498098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j" (OuterVolumeSpecName: "kube-api-access-7846j") pod "a3e9cfc1-0945-4375-a27f-7e3322e20a0a" (UID: "a3e9cfc1-0945-4375-a27f-7e3322e20a0a"). InnerVolumeSpecName "kube-api-access-7846j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:06:50.500481 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.500457    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util" (OuterVolumeSpecName: "util") pod "a3e9cfc1-0945-4375-a27f-7e3322e20a0a" (UID: "a3e9cfc1-0945-4375-a27f-7e3322e20a0a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:06:50.597292 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.597270    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:50.597292 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.597290    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7846j\" (UniqueName: \"kubernetes.io/projected/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-kube-api-access-7846j\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:50.597409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:50.597299    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e9cfc1-0945-4375-a27f-7e3322e20a0a-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:06:51.274625 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:51.274601    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt"
Jul 28 19:06:51.274789 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:51.274626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f554lt" event={"ID":"a3e9cfc1-0945-4375-a27f-7e3322e20a0a","Type":"ContainerDied","Data":"bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f"}
Jul 28 19:06:51.274789 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:51.274666    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcdf5c08e5dd5262da7d0353fdfda7abf97e3b74dcac35eb262197e84d1b5a3f"
Jul 28 19:06:54.266037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:54.266008    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cc82l"
Jul 28 19:06:55.680720 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.680690    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"]
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.680995    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="util"
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681005    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="util"
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681014    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="pull"
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681019    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="pull"
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="extract"
Jul 28 19:06:55.681077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681035    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="extract"
Jul 28 19:06:55.681318 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.681092    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e9cfc1-0945-4375-a27f-7e3322e20a0a" containerName="extract"
Jul 28 19:06:55.684698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.684683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.687229 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.687191    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:06:55.687940 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.687925    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:06:55.687997 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.687932    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xh72v\""
Jul 28 19:06:55.692535 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.692516    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"]
Jul 28 19:06:55.736091 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.736066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f7f433-db83-40bb-87e5-dbdd784b50ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.736193 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.736108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxdr8\" (UniqueName: \"kubernetes.io/projected/85f7f433-db83-40bb-87e5-dbdd784b50ee-kube-api-access-hxdr8\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.837074 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.837045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f7f433-db83-40bb-87e5-dbdd784b50ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.837182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.837104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxdr8\" (UniqueName: \"kubernetes.io/projected/85f7f433-db83-40bb-87e5-dbdd784b50ee-kube-api-access-hxdr8\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.837442 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.837422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f7f433-db83-40bb-87e5-dbdd784b50ee-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.844551 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.844531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxdr8\" (UniqueName: \"kubernetes.io/projected/85f7f433-db83-40bb-87e5-dbdd784b50ee-kube-api-access-hxdr8\") pod \"openshift-lws-operator-fd8ccff4c-8tfg9\" (UID: \"85f7f433-db83-40bb-87e5-dbdd784b50ee\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:55.994054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:55.994000    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"
Jul 28 19:06:56.117332 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:56.117256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9"]
Jul 28 19:06:56.119765 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:06:56.119723    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85f7f433_db83_40bb_87e5_dbdd784b50ee.slice/crio-137e49945991bc2b77fde1a94524f26d645f03ca0fed93cb6c3aa9453be4584f WatchSource:0}: Error finding container 137e49945991bc2b77fde1a94524f26d645f03ca0fed93cb6c3aa9453be4584f: Status 404 returned error can't find the container with id 137e49945991bc2b77fde1a94524f26d645f03ca0fed93cb6c3aa9453be4584f
Jul 28 19:06:56.299318 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:56.299248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9" event={"ID":"85f7f433-db83-40bb-87e5-dbdd784b50ee","Type":"ContainerStarted","Data":"137e49945991bc2b77fde1a94524f26d645f03ca0fed93cb6c3aa9453be4584f"}
Jul 28 19:06:59.317942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:59.317909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9" event={"ID":"85f7f433-db83-40bb-87e5-dbdd784b50ee","Type":"ContainerStarted","Data":"3a32b77a971f7ea53915040cb0a55cd145ed1b0e4aeaafd013ff5017841dea45"}
Jul 28 19:06:59.333762 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:06:59.333715    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8tfg9" podStartSLOduration=1.8299720179999999 podStartE2EDuration="4.33370183s" totalImagesPullingTime="2.503729812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:59.33326057 +0000 UTC m=+286.775181085" watchObservedRunningTime="2026-07-28 19:06:59.33370183 +0000 UTC m=+286.775622337"
Jul 28 19:07:01.637521 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.637485    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"]
Jul 28 19:07:01.641197 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.641175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.643658 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.643635    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 19:07:01.643756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.643658    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 19:07:01.644358 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.644343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vxk74\""
Jul 28 19:07:01.648795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.648777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"]
Jul 28 19:07:01.683682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.683653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.683771 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.683689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.683820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.683780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp9jq\" (UniqueName: \"kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.784948 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.784913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp9jq\" (UniqueName: \"kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.785089 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.784969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.785089 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.784990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.785393 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.785378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.785438 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.785409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.793524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.793500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp9jq\" (UniqueName: \"kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:01.951781 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:01.951703    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:02.074626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:02.074603    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"]
Jul 28 19:07:02.077010 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:02.076982    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8a45f8_ebf6_4a13_af8c_3a8fe40e2f9e.slice/crio-a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66 WatchSource:0}: Error finding container a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66: Status 404 returned error can't find the container with id a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66
Jul 28 19:07:02.332054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:02.332025    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerID="7123f987c19e4dae8fed1616fc1ca5077e2750366bf7be6f192242be6474e64c" exitCode=0
Jul 28 19:07:02.332173 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:02.332111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7" event={"ID":"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e","Type":"ContainerDied","Data":"7123f987c19e4dae8fed1616fc1ca5077e2750366bf7be6f192242be6474e64c"}
Jul 28 19:07:02.332173 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:02.332159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7" event={"ID":"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e","Type":"ContainerStarted","Data":"a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66"}
Jul 28 19:07:03.338768 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:03.338733    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerID="5f0a3dee0202f8227a8ec428e57c355fe5039da5a42dea0ccc8e4e6a87f6b73f" exitCode=0
Jul 28 19:07:03.339150 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:03.338823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7" event={"ID":"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e","Type":"ContainerDied","Data":"5f0a3dee0202f8227a8ec428e57c355fe5039da5a42dea0ccc8e4e6a87f6b73f"}
Jul 28 19:07:04.345031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:04.344996    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerID="a81e27bea0e7c1b8419e5c1830a1fbcbf8cb9ef50704f8014ba8ac7944975ea3" exitCode=0
Jul 28 19:07:04.345409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:04.345074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7" event={"ID":"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e","Type":"ContainerDied","Data":"a81e27bea0e7c1b8419e5c1830a1fbcbf8cb9ef50704f8014ba8ac7944975ea3"}
Jul 28 19:07:05.475894 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.475870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:05.616865 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.616786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util\") pod \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") "
Jul 28 19:07:05.616865 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.616831    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp9jq\" (UniqueName: \"kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq\") pod \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") "
Jul 28 19:07:05.616865 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.616854    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle\") pod \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\" (UID: \"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e\") "
Jul 28 19:07:05.617790 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.617759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle" (OuterVolumeSpecName: "bundle") pod "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" (UID: "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:05.618858 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.618837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq" (OuterVolumeSpecName: "kube-api-access-kp9jq") pod "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" (UID: "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e"). InnerVolumeSpecName "kube-api-access-kp9jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:07:05.622747 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.622725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util" (OuterVolumeSpecName: "util") pod "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" (UID: "5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:05.717892 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.717859    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:05.717892 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.717891    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp9jq\" (UniqueName: \"kubernetes.io/projected/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-kube-api-access-kp9jq\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:05.717892 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:05.717902    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:06.356107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:06.356067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7" event={"ID":"5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e","Type":"ContainerDied","Data":"a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66"}
Jul 28 19:07:06.356107 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:06.356110    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflqbx7"
Jul 28 19:07:06.356417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:06.356124    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a37b89f877051f92bf40d614802a0927b9970e637194c3af38016b4523a95c66"
Jul 28 19:07:12.883949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:12.883924    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:07:15.854308 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854275    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"]
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854598    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="extract"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854609    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="extract"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854627    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="pull"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854633    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="pull"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854654    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="util"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854660    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="util"
Jul 28 19:07:15.857443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.854714    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8a45f8-ebf6-4a13-af8c-3a8fe40e2f9e" containerName="extract"
Jul 28 19:07:15.858909 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.858890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:15.864125 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.864104    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:07:15.864250 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.864184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2sd7z\""
Jul 28 19:07:15.864693 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.864673    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:07:15.864913 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.864894    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:07:15.865048 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.865029    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:07:15.876636 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:15.876612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"]
Jul 28 19:07:16.001663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.001636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.001809 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.001672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.001809 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.001694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.001809 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.001759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbs2\" (UniqueName: \"kubernetes.io/projected/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-kube-api-access-wkbs2\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.001944 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.001850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102383 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbs2\" (UniqueName: \"kubernetes.io/projected/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-kube-api-access-wkbs2\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.102919 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.102895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.104815 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.104764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.104921 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.104823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.105062 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.105044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.110174 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.110156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbs2\" (UniqueName: \"kubernetes.io/projected/1246b0ff-4d7b-42b5-bdf3-f30f5a597482-kube-api-access-wkbs2\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-fwshq\" (UID: \"1246b0ff-4d7b-42b5-bdf3-f30f5a597482\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.169184 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.169160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:16.303637 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.303611    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"]
Jul 28 19:07:16.306130 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:16.306080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1246b0ff_4d7b_42b5_bdf3_f30f5a597482.slice/crio-b736737336afa19c9b8d866d56ce89e078da967526873855238b3422ed9793c8 WatchSource:0}: Error finding container b736737336afa19c9b8d866d56ce89e078da967526873855238b3422ed9793c8: Status 404 returned error can't find the container with id b736737336afa19c9b8d866d56ce89e078da967526873855238b3422ed9793c8
Jul 28 19:07:16.308118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.308103    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:07:16.400861 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:16.400799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq" event={"ID":"1246b0ff-4d7b-42b5-bdf3-f30f5a597482","Type":"ContainerStarted","Data":"b736737336afa19c9b8d866d56ce89e078da967526873855238b3422ed9793c8"}
Jul 28 19:07:20.421461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:20.421424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq" event={"ID":"1246b0ff-4d7b-42b5-bdf3-f30f5a597482","Type":"ContainerStarted","Data":"b804b7373736acb17c25ef77ac706ddac994c1df80cabe4f870354d8147251d4"}
Jul 28 19:07:21.427275 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:21.427242    2577 generic.go:356] "Generic (PLEG): container finished" podID="1246b0ff-4d7b-42b5-bdf3-f30f5a597482" containerID="b804b7373736acb17c25ef77ac706ddac994c1df80cabe4f870354d8147251d4" exitCode=0
Jul 28 19:07:21.427700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:21.427336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq" event={"ID":"1246b0ff-4d7b-42b5-bdf3-f30f5a597482","Type":"ContainerDied","Data":"b804b7373736acb17c25ef77ac706ddac994c1df80cabe4f870354d8147251d4"}
Jul 28 19:07:22.434488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:22.434445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq" event={"ID":"1246b0ff-4d7b-42b5-bdf3-f30f5a597482","Type":"ContainerStarted","Data":"86c8cd828ff35ccdebc3cbf7ab6e9d459a5cb2c41d1c2eaf87ecbe5bd8b56935"}
Jul 28 19:07:22.434985 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:22.434593    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:22.456141 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:22.456101    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq" podStartSLOduration=2.931887659 podStartE2EDuration="7.456088558s" totalImagesPullingTime="3.478200981s" totalInitContainerRuntime="1.045999918s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:22.454383125 +0000 UTC m=+309.896303633" watchObservedRunningTime="2026-07-28 19:07:22.456088558 +0000 UTC m=+309.898009065"
Jul 28 19:07:33.442087 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:33.442053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-fwshq"
Jul 28 19:07:35.913763 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.913725    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"]
Jul 28 19:07:35.917132 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.917110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:35.920901 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.920879    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:07:35.921006 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.920934    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gxk5p\""
Jul 28 19:07:35.921751 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.921733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 19:07:35.922018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.921992    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:07:35.922018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.922010    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:07:35.929152 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.929130    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"]
Jul 28 19:07:35.970043 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.970011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tmp\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:35.970159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.970052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgw5q\" (UniqueName: \"kubernetes.io/projected/5cbaafb5-d683-49f7-ab02-f5c0248eba67-kube-api-access-wgw5q\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:35.970159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:35.970081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.071052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.071026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tmp\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.071052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.071055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgw5q\" (UniqueName: \"kubernetes.io/projected/5cbaafb5-d683-49f7-ab02-f5c0248eba67-kube-api-access-wgw5q\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.071254 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.071076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.071254 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.071186    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 19:07:36.071331 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.071259    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs podName:5cbaafb5-d683-49f7-ab02-f5c0248eba67 nodeName:}" failed. No retries permitted until 2026-07-28 19:07:36.571242823 +0000 UTC m=+324.013163310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs") pod "kube-auth-proxy-5b4694559d-crr5f" (UID: "5cbaafb5-d683-49f7-ab02-f5c0248eba67") : secret "kube-auth-proxy-tls" not found
Jul 28 19:07:36.073329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.073306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tmp\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.091476 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.091454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgw5q\" (UniqueName: \"kubernetes.io/projected/5cbaafb5-d683-49f7-ab02-f5c0248eba67-kube-api-access-wgw5q\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.292516 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.292430    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dh7kc"]
Jul 28 19:07:36.295880 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.295865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.299057 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.299039    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 19:07:36.299761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.299741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-99hp2\""
Jul 28 19:07:36.314832 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.314807    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dh7kc"]
Jul 28 19:07:36.373452 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.373416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.373621 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.373464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp469\" (UniqueName: \"kubernetes.io/projected/983016fc-8d6c-4434-a5a7-47107a15cb8f-kube-api-access-dp469\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.474277 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.474233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.474475 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.474290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp469\" (UniqueName: \"kubernetes.io/projected/983016fc-8d6c-4434-a5a7-47107a15cb8f-kube-api-access-dp469\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.474475 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.474366    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 19:07:36.474475 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.474442    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert podName:983016fc-8d6c-4434-a5a7-47107a15cb8f nodeName:}" failed. No retries permitted until 2026-07-28 19:07:36.974425772 +0000 UTC m=+324.416346258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert") pod "odh-model-controller-7998c67dc4-dh7kc" (UID: "983016fc-8d6c-4434-a5a7-47107a15cb8f") : secret "odh-model-controller-webhook-cert" not found
Jul 28 19:07:36.490748 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.490715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp469\" (UniqueName: \"kubernetes.io/projected/983016fc-8d6c-4434-a5a7-47107a15cb8f-kube-api-access-dp469\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.575454 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.575353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.577949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.577921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbaafb5-d683-49f7-ab02-f5c0248eba67-tls-certs\") pod \"kube-auth-proxy-5b4694559d-crr5f\" (UID: \"5cbaafb5-d683-49f7-ab02-f5c0248eba67\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.826826 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.826739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"
Jul 28 19:07:36.963450 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.963422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-crr5f"]
Jul 28 19:07:36.965531 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:36.965503    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbaafb5_d683_49f7_ab02_f5c0248eba67.slice/crio-b16c556a5e63361b438da77c526294aae371d8d46e20b5fe68f1d895064856b7 WatchSource:0}: Error finding container b16c556a5e63361b438da77c526294aae371d8d46e20b5fe68f1d895064856b7: Status 404 returned error can't find the container with id b16c556a5e63361b438da77c526294aae371d8d46e20b5fe68f1d895064856b7
Jul 28 19:07:36.979116 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:36.979094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:36.979229 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.979201    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 19:07:36.979290 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:36.979281    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert podName:983016fc-8d6c-4434-a5a7-47107a15cb8f nodeName:}" failed. No retries permitted until 2026-07-28 19:07:37.979263902 +0000 UTC m=+325.421184402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert") pod "odh-model-controller-7998c67dc4-dh7kc" (UID: "983016fc-8d6c-4434-a5a7-47107a15cb8f") : secret "odh-model-controller-webhook-cert" not found
Jul 28 19:07:37.196495 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.196455    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:37.200979 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.200963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.203483 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.203438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 19:07:37.203483 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.203461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2wzxf\""
Jul 28 19:07:37.203649 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.203499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 19:07:37.203649 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.203499    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 19:07:37.203649 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.203608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 19:07:37.221086 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.221062    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:37.281369 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281544 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281544 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281544 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281544 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281720 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfp98\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.281720 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.281633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfp98\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.382726 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.382582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.383276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.383253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.384804 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.384777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.384974 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.384827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.385052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.385028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.385245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.385205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.395974 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.395950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfp98\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.401162 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.401133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7ntk5\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.504618 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.504532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f" event={"ID":"5cbaafb5-d683-49f7-ab02-f5c0248eba67","Type":"ContainerStarted","Data":"b16c556a5e63361b438da77c526294aae371d8d46e20b5fe68f1d895064856b7"}
Jul 28 19:07:37.510820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.510801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:37.656760 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.656701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:37.661014 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:37.660856    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0314e470_82ad_429d_b1fe_b276813b65df.slice/crio-8266259bd8f306645e49b8a70060392e5679f402f3e8b47f71128d8a9758ea03 WatchSource:0}: Error finding container 8266259bd8f306645e49b8a70060392e5679f402f3e8b47f71128d8a9758ea03: Status 404 returned error can't find the container with id 8266259bd8f306645e49b8a70060392e5679f402f3e8b47f71128d8a9758ea03
Jul 28 19:07:37.988252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.988192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:37.991532 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:37.991505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983016fc-8d6c-4434-a5a7-47107a15cb8f-cert\") pod \"odh-model-controller-7998c67dc4-dh7kc\" (UID: \"983016fc-8d6c-4434-a5a7-47107a15cb8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:38.106296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:38.106260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:38.241238 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:38.241195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dh7kc"]
Jul 28 19:07:38.243680 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:38.243651    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983016fc_8d6c_4434_a5a7_47107a15cb8f.slice/crio-c9812d2d62ec78835ab07832012f02ab5912f66929a9a43345577abc3547f65b WatchSource:0}: Error finding container c9812d2d62ec78835ab07832012f02ab5912f66929a9a43345577abc3547f65b: Status 404 returned error can't find the container with id c9812d2d62ec78835ab07832012f02ab5912f66929a9a43345577abc3547f65b
Jul 28 19:07:38.516533 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:38.516440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" event={"ID":"0314e470-82ad-429d-b1fe-b276813b65df","Type":"ContainerStarted","Data":"8266259bd8f306645e49b8a70060392e5679f402f3e8b47f71128d8a9758ea03"}
Jul 28 19:07:38.518050 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:38.518021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc" event={"ID":"983016fc-8d6c-4434-a5a7-47107a15cb8f","Type":"ContainerStarted","Data":"c9812d2d62ec78835ab07832012f02ab5912f66929a9a43345577abc3547f65b"}
Jul 28 19:07:42.132674 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.132642    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:07:42.133127 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.132700    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:07:42.545267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.545149    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" event={"ID":"0314e470-82ad-429d-b1fe-b276813b65df","Type":"ContainerStarted","Data":"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"}
Jul 28 19:07:42.545445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.545270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:42.546758 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.546728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc" event={"ID":"983016fc-8d6c-4434-a5a7-47107a15cb8f","Type":"ContainerStarted","Data":"e417922048bae06139cfa58b3225c10f072565c8d18b83ea605d28b9c894447c"}
Jul 28 19:07:42.546910 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.546797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:07:42.548280 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.548253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f" event={"ID":"5cbaafb5-d683-49f7-ab02-f5c0248eba67","Type":"ContainerStarted","Data":"fcf13a37c1db8d6b605035eecef1763fa57de09ad2ce31483aa139d247f4b862"}
Jul 28 19:07:42.576775 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.576716    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" podStartSLOduration=1.107556718 podStartE2EDuration="5.576694857s" totalImagesPullingTime="4.469138139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:42.571611849 +0000 UTC m=+330.013532357" watchObservedRunningTime="2026-07-28 19:07:42.576694857 +0000 UTC m=+330.018615357"
Jul 28 19:07:42.593740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:42.593676    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b4694559d-crr5f" podStartSLOduration=2.434108696 podStartE2EDuration="7.593656306s" totalImagesPullingTime="5.15954761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:42.592419767 +0000 UTC m=+330.034340277" watchObservedRunningTime="2026-07-28 19:07:42.593656306 +0000 UTC m=+330.035576814"
Jul 28 19:07:43.036322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.036244    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc" podStartSLOduration=3.097810738 podStartE2EDuration="7.036198687s" totalImagesPullingTime="3.938387949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:42.620776959 +0000 UTC m=+330.062697467" watchObservedRunningTime="2026-07-28 19:07:43.036198687 +0000 UTC m=+330.478119196"
Jul 28 19:07:43.037548 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.037521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"]
Jul 28 19:07:43.041000 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.040981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.044972 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.044950    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rtl7m\""
Jul 28 19:07:43.084661 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.084628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"]
Jul 28 19:07:43.137787 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.137751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.137815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.137855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.137891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.137971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.138014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zkf\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-kube-api-access-b8zkf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.138125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.138163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.138284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.138247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8dfa8-a03f-4174-865b-33ed4490286e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8dfa8-a03f-4174-865b-33ed4490286e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zkf\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-kube-api-access-b8zkf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.239992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.239937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.240377 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.240349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.240542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.240404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.240542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.240435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.240813 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.240788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8dfa8-a03f-4174-865b-33ed4490286e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.242116 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.242092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.242517 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.242497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.254928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.254893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.255015 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.254964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zkf\" (UniqueName: \"kubernetes.io/projected/90f8dfa8-a03f-4174-865b-33ed4490286e-kube-api-access-b8zkf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p\" (UID: \"90f8dfa8-a03f-4174-865b-33ed4490286e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.351518 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.351488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:43.489155 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.489128    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"]
Jul 28 19:07:43.490817 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:43.490786    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f8dfa8_a03f_4174_865b_33ed4490286e.slice/crio-2922291e8e83b7a8c0d98282dbd85ac3d880737d9d9aaade05f47e7013ca9f96 WatchSource:0}: Error finding container 2922291e8e83b7a8c0d98282dbd85ac3d880737d9d9aaade05f47e7013ca9f96: Status 404 returned error can't find the container with id 2922291e8e83b7a8c0d98282dbd85ac3d880737d9d9aaade05f47e7013ca9f96
Jul 28 19:07:43.554637 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.554595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p" event={"ID":"90f8dfa8-a03f-4174-865b-33ed4490286e","Type":"ContainerStarted","Data":"2922291e8e83b7a8c0d98282dbd85ac3d880737d9d9aaade05f47e7013ca9f96"}
Jul 28 19:07:43.556526 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:43.556503    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:46.305006 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.304965    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:07:46.305312 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.305053    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:07:46.305312 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.305086    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:07:46.574191 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.574062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p" event={"ID":"90f8dfa8-a03f-4174-865b-33ed4490286e","Type":"ContainerStarted","Data":"d3fb83b2e71703bf9671c926a32f258544fe307871bfc86f901a483697e55514"}
Jul 28 19:07:46.620146 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.620090    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p" podStartSLOduration=0.808220043 podStartE2EDuration="3.620075118s" totalImagesPullingTime="2.811855075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:46.619410855 +0000 UTC m=+334.061331363" watchObservedRunningTime="2026-07-28 19:07:46.620075118 +0000 UTC m=+334.061995625"
Jul 28 19:07:46.937305 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.937276    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"]
Jul 28 19:07:46.940582 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.940564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:46.943174 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.943148    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"]
Jul 28 19:07:46.944820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.944791    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 19:07:46.944820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.944797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t87tf\""
Jul 28 19:07:46.946502 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.946486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:46.948944 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.948928    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 19:07:46.953903 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.953886    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bmqrd\""
Jul 28 19:07:46.969655 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.969590    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"]
Jul 28 19:07:46.971771 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.971748    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"]
Jul 28 19:07:46.975462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.975443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqvj9\" (UniqueName: \"kubernetes.io/projected/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-kube-api-access-zqvj9\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:46.975545 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.975471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:46.975545 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.975497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-cert\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:46.975620 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:46.975564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vjx5\" (UniqueName: \"kubernetes.io/projected/7e269572-7948-4425-8422-c70da198c1ef-kube-api-access-4vjx5\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.076762 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.076728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqvj9\" (UniqueName: \"kubernetes.io/projected/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-kube-api-access-zqvj9\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:47.076762 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.076762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.076994 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.076799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-cert\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:47.076994 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:47.076970    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 19:07:47.077097 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:47.077055    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert podName:7e269572-7948-4425-8422-c70da198c1ef nodeName:}" failed. No retries permitted until 2026-07-28 19:07:47.577029824 +0000 UTC m=+335.018950330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert") pod "llmisvc-controller-manager-777c6bd7d5-m9n8w" (UID: "7e269572-7948-4425-8422-c70da198c1ef") : secret "llmisvc-webhook-server-cert" not found
Jul 28 19:07:47.077346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.077204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vjx5\" (UniqueName: \"kubernetes.io/projected/7e269572-7948-4425-8422-c70da198c1ef-kube-api-access-4vjx5\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.079326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.079297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-cert\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:47.089576 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.089552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vjx5\" (UniqueName: \"kubernetes.io/projected/7e269572-7948-4425-8422-c70da198c1ef-kube-api-access-4vjx5\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.093475 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.093456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqvj9\" (UniqueName: \"kubernetes.io/projected/cc1aafe3-5f2e-4464-805c-ae1e818c7df8-kube-api-access-zqvj9\") pod \"kserve-controller-manager-d6f4b65b8-8h4zv\" (UID: \"cc1aafe3-5f2e-4464-805c-ae1e818c7df8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:47.261177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.261088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:47.352569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.352530    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:47.358229 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.358183    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:47.421764 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:47.421733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1aafe3_5f2e_4464_805c_ae1e818c7df8.slice/crio-298e4e07fc9bf616e79e6dd2c6e22ae2f02dbf92e6b03cad687f208118824973 WatchSource:0}: Error finding container 298e4e07fc9bf616e79e6dd2c6e22ae2f02dbf92e6b03cad687f208118824973: Status 404 returned error can't find the container with id 298e4e07fc9bf616e79e6dd2c6e22ae2f02dbf92e6b03cad687f208118824973
Jul 28 19:07:47.421921 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.421896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"]
Jul 28 19:07:47.579577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.579547    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv" event={"ID":"cc1aafe3-5f2e-4464-805c-ae1e818c7df8","Type":"ContainerStarted","Data":"298e4e07fc9bf616e79e6dd2c6e22ae2f02dbf92e6b03cad687f208118824973"}
Jul 28 19:07:47.579857 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.579841    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:47.580885 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.580867    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p"
Jul 28 19:07:47.585262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.585235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.587561 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.587534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e269572-7948-4425-8422-c70da198c1ef-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m9n8w\" (UID: \"7e269572-7948-4425-8422-c70da198c1ef\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.852943 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.852855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:47.989450 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:47.989426    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"]
Jul 28 19:07:47.991819 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:07:47.991790    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7e269572_7948_4425_8422_c70da198c1ef.slice/crio-30bb4c30fd35f66c38f7963a813644748c3aa1212e276caf567c3283be0e82a4 WatchSource:0}: Error finding container 30bb4c30fd35f66c38f7963a813644748c3aa1212e276caf567c3283be0e82a4: Status 404 returned error can't find the container with id 30bb4c30fd35f66c38f7963a813644748c3aa1212e276caf567c3283be0e82a4
Jul 28 19:07:48.587739 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:48.587700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w" event={"ID":"7e269572-7948-4425-8422-c70da198c1ef","Type":"ContainerStarted","Data":"30bb4c30fd35f66c38f7963a813644748c3aa1212e276caf567c3283be0e82a4"}
Jul 28 19:07:49.609097 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:49.609063    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:49.609758 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:49.609286    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" podUID="0314e470-82ad-429d-b1fe-b276813b65df" containerName="discovery" containerID="cri-o://ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552" gracePeriod=30
Jul 28 19:07:50.173112 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.173091    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:50.215285 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfp98\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215369    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215397    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215681 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215802 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215802 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215746    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts\") pod \"0314e470-82ad-429d-b1fe-b276813b65df\" (UID: \"0314e470-82ad-429d-b1fe-b276813b65df\") "
Jul 28 19:07:50.215802 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.215771    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:07:50.216054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.216037    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-ca-configmap\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.218267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218203    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts" (OuterVolumeSpecName: "cacerts") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:07:50.218267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218247    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:07:50.218425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98" (OuterVolumeSpecName: "kube-api-access-xfp98") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "kube-api-access-xfp98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:07:50.218425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs" (OuterVolumeSpecName: "local-certs") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:50.218539 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218413    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token" (OuterVolumeSpecName: "istio-token") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:07:50.218690 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.218668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0314e470-82ad-429d-b1fe-b276813b65df" (UID: "0314e470-82ad-429d-b1fe-b276813b65df"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317428    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-csr-dns-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317453    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-istio-kubeconfig\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317463    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0314e470-82ad-429d-b1fe-b276813b65df-local-certs\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317472    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-istio-token\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317479    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0314e470-82ad-429d-b1fe-b276813b65df-cacerts\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.317504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.317487    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfp98\" (UniqueName: \"kubernetes.io/projected/0314e470-82ad-429d-b1fe-b276813b65df-kube-api-access-xfp98\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:07:50.601019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.600981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv" event={"ID":"cc1aafe3-5f2e-4464-805c-ae1e818c7df8","Type":"ContainerStarted","Data":"a8e818a13d0434a0dfda0cf3202910aafc8c6255dcaf02ff318072253e58d367"}
Jul 28 19:07:50.601189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.601048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:07:50.602225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.602183    2577 generic.go:356] "Generic (PLEG): container finished" podID="0314e470-82ad-429d-b1fe-b276813b65df" containerID="ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552" exitCode=0
Jul 28 19:07:50.602225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.602206    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"
Jul 28 19:07:50.602419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.602253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" event={"ID":"0314e470-82ad-429d-b1fe-b276813b65df","Type":"ContainerDied","Data":"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"}
Jul 28 19:07:50.602419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.602300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5" event={"ID":"0314e470-82ad-429d-b1fe-b276813b65df","Type":"ContainerDied","Data":"8266259bd8f306645e49b8a70060392e5679f402f3e8b47f71128d8a9758ea03"}
Jul 28 19:07:50.602419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.602331    2577 scope.go:117] "RemoveContainer" containerID="ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"
Jul 28 19:07:50.629333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.629247    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv" podStartSLOduration=1.976068546 podStartE2EDuration="4.629204964s" totalImagesPullingTime="2.653136418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:50.628454041 +0000 UTC m=+338.070374550" watchObservedRunningTime="2026-07-28 19:07:50.629204964 +0000 UTC m=+338.071125472"
Jul 28 19:07:50.669631 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.669595    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:50.674912 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:50.674881    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7ntk5"]
Jul 28 19:07:51.053999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.053919    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0314e470-82ad-429d-b1fe-b276813b65df" path="/var/lib/kubelet/pods/0314e470-82ad-429d-b1fe-b276813b65df/volumes"
Jul 28 19:07:51.147568 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.147432    2577 scope.go:117] "RemoveContainer" containerID="ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"
Jul 28 19:07:51.147801 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:07:51.147778    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552\": container with ID starting with ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552 not found: ID does not exist" containerID="ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"
Jul 28 19:07:51.147843 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.147812    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552"} err="failed to get container status \"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552\": rpc error: code = NotFound desc = could not find container \"ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552\": container with ID starting with ea35ffbe4dfa4d91578564e2d806c09ff62d94dc94e2b5dc8ddafd7482ebb552 not found: ID does not exist"
Jul 28 19:07:51.610229 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.610174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w" event={"ID":"7e269572-7948-4425-8422-c70da198c1ef","Type":"ContainerStarted","Data":"1c76dc1b1960268e17569c28a605a7b063a2c8516374a4a6284496b27e33463a"}
Jul 28 19:07:51.610469 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.610452    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:07:51.629186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:51.629134    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w" podStartSLOduration=2.425923735 podStartE2EDuration="5.629116757s" totalImagesPullingTime="3.203193022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:51.626670158 +0000 UTC m=+339.068590665" watchObservedRunningTime="2026-07-28 19:07:51.629116757 +0000 UTC m=+339.071037265"
Jul 28 19:07:53.583773 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:07:53.583741    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dh7kc"
Jul 28 19:08:21.615661 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:21.615628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8h4zv"
Jul 28 19:08:22.616700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:22.616668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m9n8w"
Jul 28 19:08:36.653049 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.653015    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:36.653467 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.653429    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0314e470-82ad-429d-b1fe-b276813b65df" containerName="discovery"
Jul 28 19:08:36.653467 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.653443    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0314e470-82ad-429d-b1fe-b276813b65df" containerName="discovery"
Jul 28 19:08:36.653541 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.653516    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0314e470-82ad-429d-b1fe-b276813b65df" containerName="discovery"
Jul 28 19:08:36.656674 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.656654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:36.659235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.659193    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:08:36.659993 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.659977    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6cqq5\""
Jul 28 19:08:36.660065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.659999    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:08:36.671051 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.671028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:36.701712 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.701679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh949\" (UniqueName: \"kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949\") pod \"kuadrant-operator-catalog-jdzxh\" (UID: \"bdbe63f2-8a2a-4c3b-9b52-88530786391d\") " pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:36.802731 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.802692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh949\" (UniqueName: \"kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949\") pod \"kuadrant-operator-catalog-jdzxh\" (UID: \"bdbe63f2-8a2a-4c3b-9b52-88530786391d\") " pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:36.810938 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.810906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh949\" (UniqueName: \"kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949\") pod \"kuadrant-operator-catalog-jdzxh\" (UID: \"bdbe63f2-8a2a-4c3b-9b52-88530786391d\") " pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:36.966108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:36.966026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:37.013929 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.013894    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:37.098044 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.098020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:37.100345 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:37.100317    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbe63f2_8a2a_4c3b_9b52_88530786391d.slice/crio-90d9065f565f1c6c203e9e9ee5c67a59f3b776b9a5d3246c2bcea5974bf38afc WatchSource:0}: Error finding container 90d9065f565f1c6c203e9e9ee5c67a59f3b776b9a5d3246c2bcea5974bf38afc: Status 404 returned error can't find the container with id 90d9065f565f1c6c203e9e9ee5c67a59f3b776b9a5d3246c2bcea5974bf38afc
Jul 28 19:08:37.225005 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.224928    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xb75x"]
Jul 28 19:08:37.229682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.229661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:37.236409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.236385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xb75x"]
Jul 28 19:08:37.307609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.307574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gfl6\" (UniqueName: \"kubernetes.io/projected/04b4d565-ae97-480a-98ff-f01f2724b547-kube-api-access-5gfl6\") pod \"kuadrant-operator-catalog-xb75x\" (UID: \"04b4d565-ae97-480a-98ff-f01f2724b547\") " pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:37.408703 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.408664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gfl6\" (UniqueName: \"kubernetes.io/projected/04b4d565-ae97-480a-98ff-f01f2724b547-kube-api-access-5gfl6\") pod \"kuadrant-operator-catalog-xb75x\" (UID: \"04b4d565-ae97-480a-98ff-f01f2724b547\") " pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:37.421081 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.421057    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gfl6\" (UniqueName: \"kubernetes.io/projected/04b4d565-ae97-480a-98ff-f01f2724b547-kube-api-access-5gfl6\") pod \"kuadrant-operator-catalog-xb75x\" (UID: \"04b4d565-ae97-480a-98ff-f01f2724b547\") " pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:37.540637 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.540546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:37.669343 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.669317    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xb75x"]
Jul 28 19:08:37.671683 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:37.671662    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b4d565_ae97_480a_98ff_f01f2724b547.slice/crio-dfc0576ae6a45196f0e7e89820f193e7aa45242723e23ef25c1a6a2b33a90080 WatchSource:0}: Error finding container dfc0576ae6a45196f0e7e89820f193e7aa45242723e23ef25c1a6a2b33a90080: Status 404 returned error can't find the container with id dfc0576ae6a45196f0e7e89820f193e7aa45242723e23ef25c1a6a2b33a90080
Jul 28 19:08:37.839390 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.839356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" event={"ID":"bdbe63f2-8a2a-4c3b-9b52-88530786391d","Type":"ContainerStarted","Data":"90d9065f565f1c6c203e9e9ee5c67a59f3b776b9a5d3246c2bcea5974bf38afc"}
Jul 28 19:08:37.840511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:37.840489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xb75x" event={"ID":"04b4d565-ae97-480a-98ff-f01f2724b547","Type":"ContainerStarted","Data":"dfc0576ae6a45196f0e7e89820f193e7aa45242723e23ef25c1a6a2b33a90080"}
Jul 28 19:08:39.855119 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:39.855084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xb75x" event={"ID":"04b4d565-ae97-480a-98ff-f01f2724b547","Type":"ContainerStarted","Data":"6e6014b2927c5e33474a7dd57adf8ee2a5c0850b2e569360c6695dd29a9f2b5c"}
Jul 28 19:08:39.856633 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:39.856597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" event={"ID":"bdbe63f2-8a2a-4c3b-9b52-88530786391d","Type":"ContainerStarted","Data":"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"}
Jul 28 19:08:39.856754 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:39.856648    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" podUID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" containerName="registry-server" containerID="cri-o://8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b" gracePeriod=2
Jul 28 19:08:39.870209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:39.870157    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xb75x" podStartSLOduration=0.995644758 podStartE2EDuration="2.870138341s" totalImagesPullingTime="1.874493583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:08:39.869147271 +0000 UTC m=+387.311067779" watchObservedRunningTime="2026-07-28 19:08:39.870138341 +0000 UTC m=+387.312058852"
Jul 28 19:08:39.883866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:39.883824    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" podStartSLOduration=1.442885168 podStartE2EDuration="3.883809985s" totalImagesPullingTime="2.440924817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:08:39.883068178 +0000 UTC m=+387.324988708" watchObservedRunningTime="2026-07-28 19:08:39.883809985 +0000 UTC m=+387.325730493"
Jul 28 19:08:40.102525 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.102499    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:40.133901 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.133815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh949\" (UniqueName: \"kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949\") pod \"bdbe63f2-8a2a-4c3b-9b52-88530786391d\" (UID: \"bdbe63f2-8a2a-4c3b-9b52-88530786391d\") "
Jul 28 19:08:40.135920 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.135894    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949" (OuterVolumeSpecName: "kube-api-access-vh949") pod "bdbe63f2-8a2a-4c3b-9b52-88530786391d" (UID: "bdbe63f2-8a2a-4c3b-9b52-88530786391d"). InnerVolumeSpecName "kube-api-access-vh949". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:08:40.235284 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.235253    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh949\" (UniqueName: \"kubernetes.io/projected/bdbe63f2-8a2a-4c3b-9b52-88530786391d-kube-api-access-vh949\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:40.862566 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.862525    2577 generic.go:356] "Generic (PLEG): container finished" podID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" containerID="8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b" exitCode=0
Jul 28 19:08:40.862965 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.862583    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh"
Jul 28 19:08:40.862965 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.862616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" event={"ID":"bdbe63f2-8a2a-4c3b-9b52-88530786391d","Type":"ContainerDied","Data":"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"}
Jul 28 19:08:40.862965 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.862691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdzxh" event={"ID":"bdbe63f2-8a2a-4c3b-9b52-88530786391d","Type":"ContainerDied","Data":"90d9065f565f1c6c203e9e9ee5c67a59f3b776b9a5d3246c2bcea5974bf38afc"}
Jul 28 19:08:40.862965 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.862724    2577 scope.go:117] "RemoveContainer" containerID="8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"
Jul 28 19:08:40.872472 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.872455    2577 scope.go:117] "RemoveContainer" containerID="8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"
Jul 28 19:08:40.872737 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:08:40.872719    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b\": container with ID starting with 8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b not found: ID does not exist" containerID="8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"
Jul 28 19:08:40.872788 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.872747    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b"} err="failed to get container status \"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b\": rpc error: code = NotFound desc = could not find container \"8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b\": container with ID starting with 8dc2b34bc6a7d52c99a500be38a8a8adbb5614ffd0c9be188662f4ae9c46556b not found: ID does not exist"
Jul 28 19:08:40.887174 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.887152    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:40.894083 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:40.894057    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdzxh"]
Jul 28 19:08:41.052102 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:41.052068    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" path="/var/lib/kubelet/pods/bdbe63f2-8a2a-4c3b-9b52-88530786391d/volumes"
Jul 28 19:08:47.540757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:47.540726    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:47.541305 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:47.540822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:47.561852 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:47.561828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:47.917485 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:47.917458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xb75x"
Jul 28 19:08:52.044192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.044155    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"]
Jul 28 19:08:52.044605 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.044570    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" containerName="registry-server"
Jul 28 19:08:52.044605 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.044582    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" containerName="registry-server"
Jul 28 19:08:52.044678 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.044644    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbe63f2-8a2a-4c3b-9b52-88530786391d" containerName="registry-server"
Jul 28 19:08:52.049302 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.049285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.051958 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.051944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hz72r\""
Jul 28 19:08:52.055407 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.055386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"]
Jul 28 19:08:52.139675 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.139640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.139856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.139694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.139856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.139805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q55c2\" (UniqueName: \"kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.240749 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.240712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q55c2\" (UniqueName: \"kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.240911 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.240762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.240957 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.240932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.241111 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.241094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.241272 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.241257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.249364 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.249341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q55c2\" (UniqueName: \"kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.359926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.359894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:52.502694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.502663    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"]
Jul 28 19:08:52.506094 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:52.506067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod754786e7_24c6_4943_bed5_8ad628c21cef.slice/crio-81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f WatchSource:0}: Error finding container 81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f: Status 404 returned error can't find the container with id 81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f
Jul 28 19:08:52.661700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.661618    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"]
Jul 28 19:08:52.665355 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.665327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.674830 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.674802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"]
Jul 28 19:08:52.745385 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.745352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.745542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.745426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.745542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.745483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d6sm\" (UniqueName: \"kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.846616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.846578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.846786 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.846663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.846786 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.846692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d6sm\" (UniqueName: \"kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.846944 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.846925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.846985 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.846972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.859950 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.859924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d6sm\" (UniqueName: \"kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:52.921242 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.921151    2577 generic.go:356] "Generic (PLEG): container finished" podID="754786e7-24c6-4943-bed5-8ad628c21cef" containerID="9a7f5644ddc8c00804da6f0c3faef98cd26197df05ec22521d2eb48d2a8b4465" exitCode=0
Jul 28 19:08:52.921381 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.921236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88" event={"ID":"754786e7-24c6-4943-bed5-8ad628c21cef","Type":"ContainerDied","Data":"9a7f5644ddc8c00804da6f0c3faef98cd26197df05ec22521d2eb48d2a8b4465"}
Jul 28 19:08:52.921381 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.921286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88" event={"ID":"754786e7-24c6-4943-bed5-8ad628c21cef","Type":"ContainerStarted","Data":"81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f"}
Jul 28 19:08:52.974995 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:52.974962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:53.053547 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.053520    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"]
Jul 28 19:08:53.058534 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.058511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.059498 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.059477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"]
Jul 28 19:08:53.112224 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.112191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"]
Jul 28 19:08:53.113850 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:53.113829    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5da2852_56d0_4ea1_9552_877078061f61.slice/crio-f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e WatchSource:0}: Error finding container f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e: Status 404 returned error can't find the container with id f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e
Jul 28 19:08:53.149268 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.149239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.149383 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.149348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pxcc\" (UniqueName: \"kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.149491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.149468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.249990 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.249960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pxcc\" (UniqueName: \"kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.250177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.250011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.250177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.250046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.250389 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.250369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.250434 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.250400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.259097 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.259066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pxcc\" (UniqueName: \"kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.371193 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.371150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:53.448738 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.448710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"]
Jul 28 19:08:53.453559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.453520    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.461301 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.461244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"]
Jul 28 19:08:53.507650 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.507624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"]
Jul 28 19:08:53.509801 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:53.509775    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c39614f_c24c_4a96_97fa_1ce73e8122a3.slice/crio-3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54 WatchSource:0}: Error finding container 3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54: Status 404 returned error can't find the container with id 3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54
Jul 28 19:08:53.553463 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.553435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.553577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.553496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.553577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.553543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crpfr\" (UniqueName: \"kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.655007 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.654977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.655170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.655028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.655170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.655084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crpfr\" (UniqueName: \"kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.655392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.655375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.655458 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.655411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.664390 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.664364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crpfr\" (UniqueName: \"kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.767019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.766989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:53.903886 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.903863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"]
Jul 28 19:08:53.905633 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:08:53.905605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0cbc3b7_f1a7_4690_9374_587c47cffdd9.slice/crio-474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11 WatchSource:0}: Error finding container 474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11: Status 404 returned error can't find the container with id 474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11
Jul 28 19:08:53.929149 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.929124    2577 generic.go:356] "Generic (PLEG): container finished" podID="754786e7-24c6-4943-bed5-8ad628c21cef" containerID="c4cfedd0819740bf02de80ef5778a7974debc54da5e53e0f72229e64c36a1740" exitCode=0
Jul 28 19:08:53.929276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.929208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88" event={"ID":"754786e7-24c6-4943-bed5-8ad628c21cef","Type":"ContainerDied","Data":"c4cfedd0819740bf02de80ef5778a7974debc54da5e53e0f72229e64c36a1740"}
Jul 28 19:08:53.930363 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.930339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr" event={"ID":"e0cbc3b7-f1a7-4690-9374-587c47cffdd9","Type":"ContainerStarted","Data":"474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11"}
Jul 28 19:08:53.931949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.931923    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5da2852-56d0-4ea1-9552-877078061f61" containerID="ba0ae9b0cb5b81ec1777783f7ade35ca98568dace980db0bcfa69e673ca1c25d" exitCode=0
Jul 28 19:08:53.932049 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.932013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerDied","Data":"ba0ae9b0cb5b81ec1777783f7ade35ca98568dace980db0bcfa69e673ca1c25d"}
Jul 28 19:08:53.932104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.932076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerStarted","Data":"f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e"}
Jul 28 19:08:53.933773 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.933751    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerID="a78f3e3bd6b7d98800f3283b1ee884cadde9ffcccf3b734b0f5dfa59969b3e1b" exitCode=0
Jul 28 19:08:53.933853 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.933833    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6" event={"ID":"6c39614f-c24c-4a96-97fa-1ce73e8122a3","Type":"ContainerDied","Data":"a78f3e3bd6b7d98800f3283b1ee884cadde9ffcccf3b734b0f5dfa59969b3e1b"}
Jul 28 19:08:53.933925 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:53.933870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6" event={"ID":"6c39614f-c24c-4a96-97fa-1ce73e8122a3","Type":"ContainerStarted","Data":"3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54"}
Jul 28 19:08:54.050118 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:08:54.050079    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0cbc3b7_f1a7_4690_9374_587c47cffdd9.slice/crio-e8d6667e356ea7af3bd43935effc371d0c418fc9eb3a9e65f5119cfdbedcda9d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0cbc3b7_f1a7_4690_9374_587c47cffdd9.slice/crio-conmon-e8d6667e356ea7af3bd43935effc371d0c418fc9eb3a9e65f5119cfdbedcda9d.scope\": RecentStats: unable to find data in memory cache]"
Jul 28 19:08:54.941176 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:54.941147    2577 generic.go:356] "Generic (PLEG): container finished" podID="754786e7-24c6-4943-bed5-8ad628c21cef" containerID="4aee19eb63f8ba0dced90df113f4692ac87d3fd1f2ce4d26b095a0c602eed780" exitCode=0
Jul 28 19:08:54.941563 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:54.941241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88" event={"ID":"754786e7-24c6-4943-bed5-8ad628c21cef","Type":"ContainerDied","Data":"4aee19eb63f8ba0dced90df113f4692ac87d3fd1f2ce4d26b095a0c602eed780"}
Jul 28 19:08:54.942625 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:54.942606    2577 generic.go:356] "Generic (PLEG): container finished" podID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerID="e8d6667e356ea7af3bd43935effc371d0c418fc9eb3a9e65f5119cfdbedcda9d" exitCode=0
Jul 28 19:08:54.942734 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:54.942678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr" event={"ID":"e0cbc3b7-f1a7-4690-9374-587c47cffdd9","Type":"ContainerDied","Data":"e8d6667e356ea7af3bd43935effc371d0c418fc9eb3a9e65f5119cfdbedcda9d"}
Jul 28 19:08:54.944555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:54.944512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerStarted","Data":"38acbc9854c0700f0d99b3842f1e0dee9d919643fe65b7d64a872c538a88674c"}
Jul 28 19:08:55.951182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.951094    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerID="b86942090fa6fe08de695b7ac16b1ea3e3e9ad8a69b9c5002c854b6d9fb463d3" exitCode=0
Jul 28 19:08:55.951595 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.951174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6" event={"ID":"6c39614f-c24c-4a96-97fa-1ce73e8122a3","Type":"ContainerDied","Data":"b86942090fa6fe08de695b7ac16b1ea3e3e9ad8a69b9c5002c854b6d9fb463d3"}
Jul 28 19:08:55.953163 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.953140    2577 generic.go:356] "Generic (PLEG): container finished" podID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerID="dfabcb2dd3b38cdbc50fc4ab3b537ef1efdda072019f2ac6159e86422389da66" exitCode=0
Jul 28 19:08:55.953266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.953207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr" event={"ID":"e0cbc3b7-f1a7-4690-9374-587c47cffdd9","Type":"ContainerDied","Data":"dfabcb2dd3b38cdbc50fc4ab3b537ef1efdda072019f2ac6159e86422389da66"}
Jul 28 19:08:55.955458 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.955441    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5da2852-56d0-4ea1-9552-877078061f61" containerID="38acbc9854c0700f0d99b3842f1e0dee9d919643fe65b7d64a872c538a88674c" exitCode=0
Jul 28 19:08:55.955458 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.955456    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5da2852-56d0-4ea1-9552-877078061f61" containerID="58ce9bfe44bef27301acc13a36288d50e80ca2300042ae89dfaa43ccce5b36eb" exitCode=0
Jul 28 19:08:55.955576 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.955504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerDied","Data":"38acbc9854c0700f0d99b3842f1e0dee9d919643fe65b7d64a872c538a88674c"}
Jul 28 19:08:55.955576 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:55.955533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerDied","Data":"58ce9bfe44bef27301acc13a36288d50e80ca2300042ae89dfaa43ccce5b36eb"}
Jul 28 19:08:56.093380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.093349    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:56.180187 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.180157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle\") pod \"754786e7-24c6-4943-bed5-8ad628c21cef\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") "
Jul 28 19:08:56.180326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.180287    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util\") pod \"754786e7-24c6-4943-bed5-8ad628c21cef\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") "
Jul 28 19:08:56.180326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.180319    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q55c2\" (UniqueName: \"kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2\") pod \"754786e7-24c6-4943-bed5-8ad628c21cef\" (UID: \"754786e7-24c6-4943-bed5-8ad628c21cef\") "
Jul 28 19:08:56.180724 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.180691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle" (OuterVolumeSpecName: "bundle") pod "754786e7-24c6-4943-bed5-8ad628c21cef" (UID: "754786e7-24c6-4943-bed5-8ad628c21cef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:56.182377 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.182353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2" (OuterVolumeSpecName: "kube-api-access-q55c2") pod "754786e7-24c6-4943-bed5-8ad628c21cef" (UID: "754786e7-24c6-4943-bed5-8ad628c21cef"). InnerVolumeSpecName "kube-api-access-q55c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:08:56.186127 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.186103    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util" (OuterVolumeSpecName: "util") pod "754786e7-24c6-4943-bed5-8ad628c21cef" (UID: "754786e7-24c6-4943-bed5-8ad628c21cef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:56.281466 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.281433    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:56.281466 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.281461    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/754786e7-24c6-4943-bed5-8ad628c21cef-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:56.281466 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.281472    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q55c2\" (UniqueName: \"kubernetes.io/projected/754786e7-24c6-4943-bed5-8ad628c21cef-kube-api-access-q55c2\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:56.962075 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.962049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88"
Jul 28 19:08:56.962075 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.962052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88" event={"ID":"754786e7-24c6-4943-bed5-8ad628c21cef","Type":"ContainerDied","Data":"81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f"}
Jul 28 19:08:56.962601 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.962098    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81dbf3b0c4a2b55a9a2264ee4b9a74fe5570d60b92a6160a144adedadfbf200f"
Jul 28 19:08:56.964341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.964313    2577 generic.go:356] "Generic (PLEG): container finished" podID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerID="f13372cf31bd2131cab24fd557c796808fde26950b5d8ef6035b850e6b6c715e" exitCode=0
Jul 28 19:08:56.964439 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.964383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr" event={"ID":"e0cbc3b7-f1a7-4690-9374-587c47cffdd9","Type":"ContainerDied","Data":"f13372cf31bd2131cab24fd557c796808fde26950b5d8ef6035b850e6b6c715e"}
Jul 28 19:08:56.966400 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.966378    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerID="a3bbb41a63762e646d27048ea07ace8ed24f77e539c90e6fab45b24ef103357c" exitCode=0
Jul 28 19:08:56.966512 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:56.966455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6" event={"ID":"6c39614f-c24c-4a96-97fa-1ce73e8122a3","Type":"ContainerDied","Data":"a3bbb41a63762e646d27048ea07ace8ed24f77e539c90e6fab45b24ef103357c"}
Jul 28 19:08:57.097452 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.097430    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:57.188952 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.188918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d6sm\" (UniqueName: \"kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm\") pod \"b5da2852-56d0-4ea1-9552-877078061f61\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") "
Jul 28 19:08:57.188952 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.188961    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util\") pod \"b5da2852-56d0-4ea1-9552-877078061f61\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") "
Jul 28 19:08:57.189242 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.188996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle\") pod \"b5da2852-56d0-4ea1-9552-877078061f61\" (UID: \"b5da2852-56d0-4ea1-9552-877078061f61\") "
Jul 28 19:08:57.189683 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.189649    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle" (OuterVolumeSpecName: "bundle") pod "b5da2852-56d0-4ea1-9552-877078061f61" (UID: "b5da2852-56d0-4ea1-9552-877078061f61"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:57.191057 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.191013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm" (OuterVolumeSpecName: "kube-api-access-4d6sm") pod "b5da2852-56d0-4ea1-9552-877078061f61" (UID: "b5da2852-56d0-4ea1-9552-877078061f61"). InnerVolumeSpecName "kube-api-access-4d6sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:08:57.194650 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.194617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util" (OuterVolumeSpecName: "util") pod "b5da2852-56d0-4ea1-9552-877078061f61" (UID: "b5da2852-56d0-4ea1-9552-877078061f61"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:57.290060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.289962    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d6sm\" (UniqueName: \"kubernetes.io/projected/b5da2852-56d0-4ea1-9552-877078061f61-kube-api-access-4d6sm\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:57.290060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.289997    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:57.290060 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.290009    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5da2852-56d0-4ea1-9552-877078061f61-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:57.973112 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.973084    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g"
Jul 28 19:08:57.973648 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.973106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g" event={"ID":"b5da2852-56d0-4ea1-9552-877078061f61","Type":"ContainerDied","Data":"f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e"}
Jul 28 19:08:57.973648 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:57.973161    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4f9656e9dad5ce0373e591387eccef4e6f7108a1143aa5a8349a43b695af86e"
Jul 28 19:08:58.134264 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.134240    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:58.136757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.136726    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:58.198956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.198927    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle\") pod \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") "
Jul 28 19:08:58.199128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.198990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle\") pod \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") "
Jul 28 19:08:58.199128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crpfr\" (UniqueName: \"kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr\") pod \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") "
Jul 28 19:08:58.199128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util\") pod \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") "
Jul 28 19:08:58.199128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util\") pod \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\" (UID: \"e0cbc3b7-f1a7-4690-9374-587c47cffdd9\") "
Jul 28 19:08:58.199290 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pxcc\" (UniqueName: \"kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc\") pod \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\" (UID: \"6c39614f-c24c-4a96-97fa-1ce73e8122a3\") "
Jul 28 19:08:58.199572 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199537    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle" (OuterVolumeSpecName: "bundle") pod "6c39614f-c24c-4a96-97fa-1ce73e8122a3" (UID: "6c39614f-c24c-4a96-97fa-1ce73e8122a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:58.199756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.199731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle" (OuterVolumeSpecName: "bundle") pod "e0cbc3b7-f1a7-4690-9374-587c47cffdd9" (UID: "e0cbc3b7-f1a7-4690-9374-587c47cffdd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:58.201245 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.201200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc" (OuterVolumeSpecName: "kube-api-access-5pxcc") pod "6c39614f-c24c-4a96-97fa-1ce73e8122a3" (UID: "6c39614f-c24c-4a96-97fa-1ce73e8122a3"). InnerVolumeSpecName "kube-api-access-5pxcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:08:58.201389 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.201369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr" (OuterVolumeSpecName: "kube-api-access-crpfr") pod "e0cbc3b7-f1a7-4690-9374-587c47cffdd9" (UID: "e0cbc3b7-f1a7-4690-9374-587c47cffdd9"). InnerVolumeSpecName "kube-api-access-crpfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:08:58.204893 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.204865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util" (OuterVolumeSpecName: "util") pod "e0cbc3b7-f1a7-4690-9374-587c47cffdd9" (UID: "e0cbc3b7-f1a7-4690-9374-587c47cffdd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:58.205126 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.205105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util" (OuterVolumeSpecName: "util") pod "6c39614f-c24c-4a96-97fa-1ce73e8122a3" (UID: "6c39614f-c24c-4a96-97fa-1ce73e8122a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300403    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300451    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-util\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300462    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pxcc\" (UniqueName: \"kubernetes.io/projected/6c39614f-c24c-4a96-97fa-1ce73e8122a3-kube-api-access-5pxcc\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300472    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300480    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c39614f-c24c-4a96-97fa-1ce73e8122a3-bundle\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.300505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.300488    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crpfr\" (UniqueName: \"kubernetes.io/projected/e0cbc3b7-f1a7-4690-9374-587c47cffdd9-kube-api-access-crpfr\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:08:58.979433 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.979409    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6"
Jul 28 19:08:58.979862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.979439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6" event={"ID":"6c39614f-c24c-4a96-97fa-1ce73e8122a3","Type":"ContainerDied","Data":"3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54"}
Jul 28 19:08:58.979862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.979511    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3933c3bd95c4d584ac1058eb8d88fb35d779e79601c8d62895d2cc6da7f36a54"
Jul 28 19:08:58.981431 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.981407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr" event={"ID":"e0cbc3b7-f1a7-4690-9374-587c47cffdd9","Type":"ContainerDied","Data":"474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11"}
Jul 28 19:08:58.981431 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.981430    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr"
Jul 28 19:08:58.981617 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:08:58.981447    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="474f8ec3e8f15ab3495fe277647534724d456d13ca496b70982846583782af11"
Jul 28 19:09:08.221575 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221540    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"]
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="util"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="util"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221909    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="extract"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221914    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="extract"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221920    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="extract"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221927    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="extract"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221936    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="pull"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221941    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="pull"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221949    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="util"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221954    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="util"
Jul 28 19:09:08.221956 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221961    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221966    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221972    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221977    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221984    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="util"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221989    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="util"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.221998    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="util"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222002    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="util"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222009    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222013    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222026    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222047    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="pull"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222099    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c39614f-c24c-4a96-97fa-1ce73e8122a3" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222108    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="754786e7-24c6-4943-bed5-8ad628c21cef" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222114    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5da2852-56d0-4ea1-9552-877078061f61" containerName="extract"
Jul 28 19:09:08.222296 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.222121    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0cbc3b7-f1a7-4690-9374-587c47cffdd9" containerName="extract"
Jul 28 19:09:08.225079 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.225063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:08.228671 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.228641    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-46wxx\""
Jul 28 19:09:08.249380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.249350    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"]
Jul 28 19:09:08.390758 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.390720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtp2f\" (UniqueName: \"kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f\") pod \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" (UID: \"15eb1d88-9bad-4146-ab74-e5ae3994bb0d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:08.492305 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.492187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtp2f\" (UniqueName: \"kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f\") pod \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" (UID: \"15eb1d88-9bad-4146-ab74-e5ae3994bb0d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:08.507023 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.506997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtp2f\" (UniqueName: \"kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f\") pod \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" (UID: \"15eb1d88-9bad-4146-ab74-e5ae3994bb0d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:08.534957 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.534926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:08.671398 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:08.671368    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"]
Jul 28 19:09:08.673307 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:08.673281    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15eb1d88_9bad_4146_ab74_e5ae3994bb0d.slice/crio-110670b5c16698b600e5a155cc508c1f1f06d5e26f666dcd40dee2ac43f0a3cc WatchSource:0}: Error finding container 110670b5c16698b600e5a155cc508c1f1f06d5e26f666dcd40dee2ac43f0a3cc: Status 404 returned error can't find the container with id 110670b5c16698b600e5a155cc508c1f1f06d5e26f666dcd40dee2ac43f0a3cc
Jul 28 19:09:09.044758 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:09.044720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq" event={"ID":"15eb1d88-9bad-4146-ab74-e5ae3994bb0d","Type":"ContainerStarted","Data":"110670b5c16698b600e5a155cc508c1f1f06d5e26f666dcd40dee2ac43f0a3cc"}
Jul 28 19:09:11.058458 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:11.058372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq" event={"ID":"15eb1d88-9bad-4146-ab74-e5ae3994bb0d","Type":"ContainerStarted","Data":"b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1"}
Jul 28 19:09:11.058839 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:11.058495    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:11.083460 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:11.083412    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq" podStartSLOduration=1.13401436 podStartE2EDuration="3.083397855s" totalImagesPullingTime="1.949383495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:11.081863256 +0000 UTC m=+418.523783775" watchObservedRunningTime="2026-07-28 19:09:11.083397855 +0000 UTC m=+418.525318362"
Jul 28 19:09:12.296319 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.296280    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h662z"]
Jul 28 19:09:12.301030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.301015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:12.305194 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.305169    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rmvvl\""
Jul 28 19:09:12.326870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.326841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h662z"]
Jul 28 19:09:12.431710 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.431667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wrf\" (UniqueName: \"kubernetes.io/projected/141fde96-b7d5-4b72-8644-5b858660a319-kube-api-access-t7wrf\") pod \"authorino-operator-75cc8d5cb7-h662z\" (UID: \"141fde96-b7d5-4b72-8644-5b858660a319\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:12.532480 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.532424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wrf\" (UniqueName: \"kubernetes.io/projected/141fde96-b7d5-4b72-8644-5b858660a319-kube-api-access-t7wrf\") pod \"authorino-operator-75cc8d5cb7-h662z\" (UID: \"141fde96-b7d5-4b72-8644-5b858660a319\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:12.544397 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.544370    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wrf\" (UniqueName: \"kubernetes.io/projected/141fde96-b7d5-4b72-8644-5b858660a319-kube-api-access-t7wrf\") pod \"authorino-operator-75cc8d5cb7-h662z\" (UID: \"141fde96-b7d5-4b72-8644-5b858660a319\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:12.611351 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.611317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:12.756983 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:12.756958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h662z"]
Jul 28 19:09:12.759742 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:12.759714    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141fde96_b7d5_4b72_8644_5b858660a319.slice/crio-b0ef689a506cbdf573845adc5f4e0e02701b9e544314abb66ee42c5b9febfbdc WatchSource:0}: Error finding container b0ef689a506cbdf573845adc5f4e0e02701b9e544314abb66ee42c5b9febfbdc: Status 404 returned error can't find the container with id b0ef689a506cbdf573845adc5f4e0e02701b9e544314abb66ee42c5b9febfbdc
Jul 28 19:09:13.068984 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:13.068951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z" event={"ID":"141fde96-b7d5-4b72-8644-5b858660a319","Type":"ContainerStarted","Data":"b0ef689a506cbdf573845adc5f4e0e02701b9e544314abb66ee42c5b9febfbdc"}
Jul 28 19:09:15.734372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.734332    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"]
Jul 28 19:09:15.737910 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.737893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:15.741548 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.741530    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 19:09:15.741640 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.741548    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xjkks\""
Jul 28 19:09:15.752694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.752670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"]
Jul 28 19:09:15.757856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.757832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hpt\" (UniqueName: \"kubernetes.io/projected/31d2cef2-3c1f-46cf-ba48-b82e67777b4e-kube-api-access-k2hpt\") pod \"dns-operator-controller-manager-864dd78c4c-7v42v\" (UID: \"31d2cef2-3c1f-46cf-ba48-b82e67777b4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:15.858322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.858284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hpt\" (UniqueName: \"kubernetes.io/projected/31d2cef2-3c1f-46cf-ba48-b82e67777b4e-kube-api-access-k2hpt\") pod \"dns-operator-controller-manager-864dd78c4c-7v42v\" (UID: \"31d2cef2-3c1f-46cf-ba48-b82e67777b4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:15.869894 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:15.869858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hpt\" (UniqueName: \"kubernetes.io/projected/31d2cef2-3c1f-46cf-ba48-b82e67777b4e-kube-api-access-k2hpt\") pod \"dns-operator-controller-manager-864dd78c4c-7v42v\" (UID: \"31d2cef2-3c1f-46cf-ba48-b82e67777b4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:16.048411 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:16.048315    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:16.087046 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:16.087009    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z" event={"ID":"141fde96-b7d5-4b72-8644-5b858660a319","Type":"ContainerStarted","Data":"3a9acc43523831bd7fb92a32db3f861b289897cf41ca1388d226a3d4040531b0"}
Jul 28 19:09:16.087225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:16.087192    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:16.211895 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:16.211849    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z" podStartSLOduration=1.93482611 podStartE2EDuration="4.211831137s" totalImagesPullingTime="2.277005027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:16.123769685 +0000 UTC m=+423.565690193" watchObservedRunningTime="2026-07-28 19:09:16.211831137 +0000 UTC m=+423.653751645"
Jul 28 19:09:16.212197 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:16.212179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"]
Jul 28 19:09:16.214582 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:16.214556    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d2cef2_3c1f_46cf_ba48_b82e67777b4e.slice/crio-2e83c14c3fec8e1ef79c92320e4f886d0864e1ff999089453ef1bd0814dc5839 WatchSource:0}: Error finding container 2e83c14c3fec8e1ef79c92320e4f886d0864e1ff999089453ef1bd0814dc5839: Status 404 returned error can't find the container with id 2e83c14c3fec8e1ef79c92320e4f886d0864e1ff999089453ef1bd0814dc5839
Jul 28 19:09:17.095044 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:17.095008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v" event={"ID":"31d2cef2-3c1f-46cf-ba48-b82e67777b4e","Type":"ContainerStarted","Data":"2e83c14c3fec8e1ef79c92320e4f886d0864e1ff999089453ef1bd0814dc5839"}
Jul 28 19:09:19.108056 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:19.108013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v" event={"ID":"31d2cef2-3c1f-46cf-ba48-b82e67777b4e","Type":"ContainerStarted","Data":"638b2793ebdb17828efd712a9c827693313a552a793ee67843933eee99a04924"}
Jul 28 19:09:19.108459 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:19.108109    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:19.155336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:19.155288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v" podStartSLOduration=2.157216607 podStartE2EDuration="4.155273647s" totalImagesPullingTime="1.99805704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:19.15394367 +0000 UTC m=+426.595864179" watchObservedRunningTime="2026-07-28 19:09:19.155273647 +0000 UTC m=+426.597194156"
Jul 28 19:09:21.115848 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.115814    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"]
Jul 28 19:09:21.119556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.119537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.122432 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.122415    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6s92b\""
Jul 28 19:09:21.143982 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.143951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"]
Jul 28 19:09:21.202385 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.202344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.202584 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.202396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcx6r\" (UniqueName: \"kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.303181 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.303142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.303386 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.303195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcx6r\" (UniqueName: \"kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.303626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.303600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.318078 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.318057    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcx6r\" (UniqueName: \"kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.429313 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.429208    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:21.589234 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:21.589195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"]
Jul 28 19:09:21.591414 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:21.591387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821b0519_cd25_4759_87d7_dcea979458b3.slice/crio-ee4f63715822d0db6480af5a51b4b7841de6dada6a7bca079ee5998f9abdd3f3 WatchSource:0}: Error finding container ee4f63715822d0db6480af5a51b4b7841de6dada6a7bca079ee5998f9abdd3f3: Status 404 returned error can't find the container with id ee4f63715822d0db6480af5a51b4b7841de6dada6a7bca079ee5998f9abdd3f3
Jul 28 19:09:22.065572 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:22.065543    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:22.126907 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:22.126875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6" event={"ID":"821b0519-cd25-4759-87d7-dcea979458b3","Type":"ContainerStarted","Data":"ee4f63715822d0db6480af5a51b4b7841de6dada6a7bca079ee5998f9abdd3f3"}
Jul 28 19:09:27.097012 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:27.096974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h662z"
Jul 28 19:09:27.160416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:27.160331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6" event={"ID":"821b0519-cd25-4759-87d7-dcea979458b3","Type":"ContainerStarted","Data":"12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0"}
Jul 28 19:09:27.160416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:27.160385    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:27.234511 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:27.234454    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6" podStartSLOduration=0.945778679 podStartE2EDuration="6.234438093s" totalImagesPullingTime="5.288659414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:27.23397792 +0000 UTC m=+434.675898445" watchObservedRunningTime="2026-07-28 19:09:27.234438093 +0000 UTC m=+434.676358600"
Jul 28 19:09:29.724737 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.724699    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"]
Jul 28 19:09:29.728295 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.728272    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.731251 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.731205    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 19:09:29.731363 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.731267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hz72r\""
Jul 28 19:09:29.731363 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.731227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 19:09:29.743269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.743247    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"]
Jul 28 19:09:29.777726 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.777695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95qg\" (UniqueName: \"kubernetes.io/projected/a86f8dc9-b711-4d33-8a03-193f8fb5d902-kube-api-access-d95qg\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.777912 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.777743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a86f8dc9-b711-4d33-8a03-193f8fb5d902-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.777912 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.777854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a86f8dc9-b711-4d33-8a03-193f8fb5d902-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.878941 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.878912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a86f8dc9-b711-4d33-8a03-193f8fb5d902-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.879118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.878958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95qg\" (UniqueName: \"kubernetes.io/projected/a86f8dc9-b711-4d33-8a03-193f8fb5d902-kube-api-access-d95qg\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.879118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.878991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a86f8dc9-b711-4d33-8a03-193f8fb5d902-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.879715 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.879695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a86f8dc9-b711-4d33-8a03-193f8fb5d902-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.881467 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.881441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a86f8dc9-b711-4d33-8a03-193f8fb5d902-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:29.892257 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:29.892236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95qg\" (UniqueName: \"kubernetes.io/projected/a86f8dc9-b711-4d33-8a03-193f8fb5d902-kube-api-access-d95qg\") pod \"kuadrant-console-plugin-66b5774985-m9j7c\" (UID: \"a86f8dc9-b711-4d33-8a03-193f8fb5d902\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:30.037586 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:30.037497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"
Jul 28 19:09:30.116628 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:30.116600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7v42v"
Jul 28 19:09:30.175434 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:30.175411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c"]
Jul 28 19:09:30.178044 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:30.178015    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda86f8dc9_b711_4d33_8a03_193f8fb5d902.slice/crio-842db8070f0474c477958415b375809ae918d9d3d44b561acdd8ebb9835f1aa9 WatchSource:0}: Error finding container 842db8070f0474c477958415b375809ae918d9d3d44b561acdd8ebb9835f1aa9: Status 404 returned error can't find the container with id 842db8070f0474c477958415b375809ae918d9d3d44b561acdd8ebb9835f1aa9
Jul 28 19:09:31.190873 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:31.190829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c" event={"ID":"a86f8dc9-b711-4d33-8a03-193f8fb5d902","Type":"ContainerStarted","Data":"842db8070f0474c477958415b375809ae918d9d3d44b561acdd8ebb9835f1aa9"}
Jul 28 19:09:38.173663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:38.173580    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:39.969578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:39.969537    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"]
Jul 28 19:09:39.970057 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:39.969802    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6" podUID="821b0519-cd25-4759-87d7-dcea979458b3" containerName="manager" containerID="cri-o://12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0" gracePeriod=2
Jul 28 19:09:39.979503 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:39.979473    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"]
Jul 28 19:09:40.003691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.003646    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"]
Jul 28 19:09:40.003929 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.003899    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" containerName="manager" containerID="cri-o://b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1" gracePeriod=2
Jul 28 19:09:40.007421 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:40.007377    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:40.010419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.010361    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"]
Jul 28 19:09:40.023526 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.023380    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:40.023960 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.023938    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="821b0519-cd25-4759-87d7-dcea979458b3" containerName="manager"
Jul 28 19:09:40.023960 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.023960    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="821b0519-cd25-4759-87d7-dcea979458b3" containerName="manager"
Jul 28 19:09:40.024118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.023988    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" containerName="manager"
Jul 28 19:09:40.024118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.023997    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" containerName="manager"
Jul 28 19:09:40.024118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.024077    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" containerName="manager"
Jul 28 19:09:40.024118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.024089    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="821b0519-cd25-4759-87d7-dcea979458b3" containerName="manager"
Jul 28 19:09:40.027505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.027483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.034428 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:40.034397    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:40.048038 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.047954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:40.052289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.052250    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"]
Jul 28 19:09:40.056461 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.056443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:40.083108 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:40.083062    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:40.087597 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.087566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"]
Jul 28 19:09:40.104605 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:40.104573    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:40.107900 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:40.107869    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:40.174712 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.174667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r6n7\" (UniqueName: \"kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.174902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.174776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.174902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.174896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4nl9\" (UniqueName: \"kubernetes.io/projected/3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe-kube-api-access-p4nl9\") pod \"limitador-operator-controller-manager-859d7b78bd-b45xm\" (UID: \"3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:40.275453 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.275358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.275616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.275460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4nl9\" (UniqueName: \"kubernetes.io/projected/3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe-kube-api-access-p4nl9\") pod \"limitador-operator-controller-manager-859d7b78bd-b45xm\" (UID: \"3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:40.275616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.275527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r6n7\" (UniqueName: \"kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.275873 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.275783    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.294636 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.294601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4nl9\" (UniqueName: \"kubernetes.io/projected/3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe-kube-api-access-p4nl9\") pod \"limitador-operator-controller-manager-859d7b78bd-b45xm\" (UID: \"3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:40.299142 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.299117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r6n7\" (UniqueName: \"kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f4vx7\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.395150 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.395117    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:40.403459 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:40.403424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:54.127606 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.127459    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.130174 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.130077    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.137094 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.137051    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:54.139539 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.139505    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:54.141465 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.141422    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.160462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.160429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"]
Jul 28 19:09:54.163856 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:54.163779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4f8ac9_2cff_4b5c_89fc_9fa6b356eafe.slice/crio-8d4f987a559603335fe0649be0784fd2a2ee02bbb718973d70e891a425c4e02e WatchSource:0}: Error finding container 8d4f987a559603335fe0649be0784fd2a2ee02bbb718973d70e891a425c4e02e: Status 404 returned error can't find the container with id 8d4f987a559603335fe0649be0784fd2a2ee02bbb718973d70e891a425c4e02e
Jul 28 19:09:54.189563 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.189481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:54.196363 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:09:54.196341    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8744cd6_2135_4977_a3ac_447d483bf9c2.slice/crio-aa0752e141e90d0a70e188aa1aa4a4937724083ae61eb3faf3e653afdbecfa9a WatchSource:0}: Error finding container aa0752e141e90d0a70e188aa1aa4a4937724083ae61eb3faf3e653afdbecfa9a: Status 404 returned error can't find the container with id aa0752e141e90d0a70e188aa1aa4a4937724083ae61eb3faf3e653afdbecfa9a
Jul 28 19:09:54.312209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.312172    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume\") pod \"821b0519-cd25-4759-87d7-dcea979458b3\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") "
Jul 28 19:09:54.312448 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.312278    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcx6r\" (UniqueName: \"kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r\") pod \"821b0519-cd25-4759-87d7-dcea979458b3\" (UID: \"821b0519-cd25-4759-87d7-dcea979458b3\") "
Jul 28 19:09:54.312448 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.312318    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtp2f\" (UniqueName: \"kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f\") pod \"15eb1d88-9bad-4146-ab74-e5ae3994bb0d\" (UID: \"15eb1d88-9bad-4146-ab74-e5ae3994bb0d\") "
Jul 28 19:09:54.312651 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.312626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "821b0519-cd25-4759-87d7-dcea979458b3" (UID: "821b0519-cd25-4759-87d7-dcea979458b3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:09:54.314329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.314307    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r" (OuterVolumeSpecName: "kube-api-access-gcx6r") pod "821b0519-cd25-4759-87d7-dcea979458b3" (UID: "821b0519-cd25-4759-87d7-dcea979458b3"). InnerVolumeSpecName "kube-api-access-gcx6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:09:54.314398 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.314337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f" (OuterVolumeSpecName: "kube-api-access-vtp2f") pod "15eb1d88-9bad-4146-ab74-e5ae3994bb0d" (UID: "15eb1d88-9bad-4146-ab74-e5ae3994bb0d"). InnerVolumeSpecName "kube-api-access-vtp2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:09:54.342098 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.342067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm" event={"ID":"3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe","Type":"ContainerStarted","Data":"86359c053e2ed652cab3e3bc8f40a14d1df9373c18591ec313080d1659277c98"}
Jul 28 19:09:54.342258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.342157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:09:54.342258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.342182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm" event={"ID":"3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe","Type":"ContainerStarted","Data":"8d4f987a559603335fe0649be0784fd2a2ee02bbb718973d70e891a425c4e02e"}
Jul 28 19:09:54.343715 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.343688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" event={"ID":"e8744cd6-2135-4977-a3ac-447d483bf9c2","Type":"ContainerStarted","Data":"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"}
Jul 28 19:09:54.343814 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.343739    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:54.343814 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.343755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" event={"ID":"e8744cd6-2135-4977-a3ac-447d483bf9c2","Type":"ContainerStarted","Data":"aa0752e141e90d0a70e188aa1aa4a4937724083ae61eb3faf3e653afdbecfa9a"}
Jul 28 19:09:54.344851 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.344829    2577 generic.go:356] "Generic (PLEG): container finished" podID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" containerID="b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1" exitCode=0
Jul 28 19:09:54.344945 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.344880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.344945 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.344915    2577 scope.go:117] "RemoveContainer" containerID="b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1"
Jul 28 19:09:54.346259 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.346235    2577 generic.go:356] "Generic (PLEG): container finished" podID="821b0519-cd25-4759-87d7-dcea979458b3" containerID="12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0" exitCode=0
Jul 28 19:09:54.346368 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.346282    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:54.348015 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.347994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c" event={"ID":"a86f8dc9-b711-4d33-8a03-193f8fb5d902","Type":"ContainerStarted","Data":"52c3fc7fee1a6c344a7b93ee26b08a2329f0bf51cad4aa5649824af8fe122325"}
Jul 28 19:09:54.355018 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.355006    2577 scope.go:117] "RemoveContainer" containerID="b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1"
Jul 28 19:09:54.355350 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.355323    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1\": container with ID starting with b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1 not found: ID does not exist" containerID="b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1"
Jul 28 19:09:54.355437 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.355353    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1"} err="failed to get container status \"b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1\": rpc error: code = NotFound desc = could not find container \"b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1\": container with ID starting with b8ad77d58fddad153dd1ba6a86498b2a2bd690cb8de68c09d73191f2b92eafa1 not found: ID does not exist"
Jul 28 19:09:54.355437 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.355369    2577 scope.go:117] "RemoveContainer" containerID="12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0"
Jul 28 19:09:54.363620 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.363580    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm" podStartSLOduration=14.363567987 podStartE2EDuration="14.363567987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:54.361630351 +0000 UTC m=+461.803550856" watchObservedRunningTime="2026-07-28 19:09:54.363567987 +0000 UTC m=+461.805488555"
Jul 28 19:09:54.363811 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.363755    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:54.363910 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.363889    2577 scope.go:117] "RemoveContainer" containerID="12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0"
Jul 28 19:09:54.364183 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.364164    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0\": container with ID starting with 12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0 not found: ID does not exist" containerID="12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0"
Jul 28 19:09:54.364255 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.364190    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0"} err="failed to get container status \"12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0\": rpc error: code = NotFound desc = could not find container \"12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0\": container with ID starting with 12a3aec3db0dc9e36531ffc4f61f0392afcca0286e00e302171a29b25b53cdf0 not found: ID does not exist"
Jul 28 19:09:54.365540 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.365518    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.367474 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.367455    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cv9m6\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="821b0519-cd25-4759-87d7-dcea979458b3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cv9m6"
Jul 28 19:09:54.369269 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:54.369244    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qfxlq\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qfxlq"
Jul 28 19:09:54.390336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.390285    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m9j7c" podStartSLOduration=1.468823925 podStartE2EDuration="25.390267864s" totalImagesPullingTime="23.921443939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:54.387776133 +0000 UTC m=+461.829696653" watchObservedRunningTime="2026-07-28 19:09:54.390267864 +0000 UTC m=+461.832188376"
Jul 28 19:09:54.413321 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.413272    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" podStartSLOduration=15.413256554 podStartE2EDuration="15.413256554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:54.408875029 +0000 UTC m=+461.850795531" watchObservedRunningTime="2026-07-28 19:09:54.413256554 +0000 UTC m=+461.855177061"
Jul 28 19:09:54.413524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.413496    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtp2f\" (UniqueName: \"kubernetes.io/projected/15eb1d88-9bad-4146-ab74-e5ae3994bb0d-kube-api-access-vtp2f\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:09:54.413524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.413529    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/821b0519-cd25-4759-87d7-dcea979458b3-extensions-socket-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:09:54.413642 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:54.413543    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcx6r\" (UniqueName: \"kubernetes.io/projected/821b0519-cd25-4759-87d7-dcea979458b3-kube-api-access-gcx6r\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:09:55.052959 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:55.052925    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15eb1d88-9bad-4146-ab74-e5ae3994bb0d" path="/var/lib/kubelet/pods/15eb1d88-9bad-4146-ab74-e5ae3994bb0d/volumes"
Jul 28 19:09:55.053300 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:55.053286    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="821b0519-cd25-4759-87d7-dcea979458b3" path="/var/lib/kubelet/pods/821b0519-cd25-4759-87d7-dcea979458b3/volumes"
Jul 28 19:09:56.657820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.657782    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:56.658271 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.657961    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" podUID="e8744cd6-2135-4977-a3ac-447d483bf9c2" containerName="manager" containerID="cri-o://440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa" gracePeriod=10
Jul 28 19:09:56.911418 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.911358    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:56.938525 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.938471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r6n7\" (UniqueName: \"kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7\") pod \"e8744cd6-2135-4977-a3ac-447d483bf9c2\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") "
Jul 28 19:09:56.938674 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.938613    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume\") pod \"e8744cd6-2135-4977-a3ac-447d483bf9c2\" (UID: \"e8744cd6-2135-4977-a3ac-447d483bf9c2\") "
Jul 28 19:09:56.939005 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.938967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e8744cd6-2135-4977-a3ac-447d483bf9c2" (UID: "e8744cd6-2135-4977-a3ac-447d483bf9c2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:09:56.940519 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:56.940494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7" (OuterVolumeSpecName: "kube-api-access-8r6n7") pod "e8744cd6-2135-4977-a3ac-447d483bf9c2" (UID: "e8744cd6-2135-4977-a3ac-447d483bf9c2"). InnerVolumeSpecName "kube-api-access-8r6n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:09:57.039574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.039544    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8744cd6-2135-4977-a3ac-447d483bf9c2-extensions-socket-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:09:57.039574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.039571    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r6n7\" (UniqueName: \"kubernetes.io/projected/e8744cd6-2135-4977-a3ac-447d483bf9c2-kube-api-access-8r6n7\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:09:57.368053 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.368012    2577 generic.go:356] "Generic (PLEG): container finished" podID="e8744cd6-2135-4977-a3ac-447d483bf9c2" containerID="440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa" exitCode=0
Jul 28 19:09:57.368192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.368065    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"
Jul 28 19:09:57.368192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.368087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" event={"ID":"e8744cd6-2135-4977-a3ac-447d483bf9c2","Type":"ContainerDied","Data":"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"}
Jul 28 19:09:57.368192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.368129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7" event={"ID":"e8744cd6-2135-4977-a3ac-447d483bf9c2","Type":"ContainerDied","Data":"aa0752e141e90d0a70e188aa1aa4a4937724083ae61eb3faf3e653afdbecfa9a"}
Jul 28 19:09:57.368192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.368150    2577 scope.go:117] "RemoveContainer" containerID="440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"
Jul 28 19:09:57.377153 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.377125    2577 scope.go:117] "RemoveContainer" containerID="440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"
Jul 28 19:09:57.377457 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:09:57.377438    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa\": container with ID starting with 440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa not found: ID does not exist" containerID="440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"
Jul 28 19:09:57.377523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.377464    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa"} err="failed to get container status \"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa\": rpc error: code = NotFound desc = could not find container \"440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa\": container with ID starting with 440d1cf22704b64fd3190b28182e6be8c94f8c733046a2a54c6fe86262d117aa not found: ID does not exist"
Jul 28 19:09:57.396191 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.396166    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:57.402779 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:57.402753    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f4vx7"]
Jul 28 19:09:59.052747 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:09:59.052713    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8744cd6-2135-4977-a3ac-447d483bf9c2" path="/var/lib/kubelet/pods/e8744cd6-2135-4977-a3ac-447d483bf9c2/volumes"
Jul 28 19:10:05.357776 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:05.357744    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-b45xm"
Jul 28 19:10:12.983850 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:12.983820    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"]
Jul 28 19:10:12.984287 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:12.984263    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8744cd6-2135-4977-a3ac-447d483bf9c2" containerName="manager"
Jul 28 19:10:12.984287 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:12.984277    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8744cd6-2135-4977-a3ac-447d483bf9c2" containerName="manager"
Jul 28 19:10:12.984420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:12.984356    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8744cd6-2135-4977-a3ac-447d483bf9c2" containerName="manager"
Jul 28 19:10:13.048614 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.048583    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.051579 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.051556    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-trhkq\""
Jul 28 19:10:13.054745 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.054728    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"]
Jul 28 19:10:13.067422 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d287db-9f2e-4137-9552-18b78a231d2b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d287db-9f2e-4137-9552-18b78a231d2b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067611 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067611 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067611 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.067873 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.067793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89psz\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-kube-api-access-89psz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169723 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169787 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89psz\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-kube-api-access-89psz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169787 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.169959 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.169926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.170042 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.170002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.170042 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.170019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d287db-9f2e-4137-9552-18b78a231d2b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.170353 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.170298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.170353 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.170343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d287db-9f2e-4137-9552-18b78a231d2b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.170999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.170973    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d287db-9f2e-4137-9552-18b78a231d2b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.171834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.171815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d287db-9f2e-4137-9552-18b78a231d2b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.172445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.172420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d287db-9f2e-4137-9552-18b78a231d2b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.177465 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.177444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.177652 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.177635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89psz\" (UniqueName: \"kubernetes.io/projected/61d287db-9f2e-4137-9552-18b78a231d2b-kube-api-access-89psz\") pod \"maas-default-gateway-openshift-default-78bb78ff78-lrtmn\" (UID: \"61d287db-9f2e-4137-9552-18b78a231d2b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.359813 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.359767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:13.515946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.515920    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"]
Jul 28 19:10:13.518025 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:13.517992    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d287db_9f2e_4137_9552_18b78a231d2b.slice/crio-28271fc140218754fa9006be71a210e159f88435e177a239bd97dc4eaed1327b WatchSource:0}: Error finding container 28271fc140218754fa9006be71a210e159f88435e177a239bd97dc4eaed1327b: Status 404 returned error can't find the container with id 28271fc140218754fa9006be71a210e159f88435e177a239bd97dc4eaed1327b
Jul 28 19:10:13.520328 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.520295    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:10:13.520429 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.520395    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:10:13.520497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:13.520438    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:10:14.467667 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:14.467625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" event={"ID":"61d287db-9f2e-4137-9552-18b78a231d2b","Type":"ContainerStarted","Data":"a3481f9e0518b5d25b80f4dc3c8fe4c0ae7a5648c3b2ea90f6327b73379c9085"}
Jul 28 19:10:14.468074 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:14.467709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" event={"ID":"61d287db-9f2e-4137-9552-18b78a231d2b","Type":"ContainerStarted","Data":"28271fc140218754fa9006be71a210e159f88435e177a239bd97dc4eaed1327b"}
Jul 28 19:10:14.487534 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:14.487479    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" podStartSLOduration=2.487463557 podStartE2EDuration="2.487463557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:14.486878252 +0000 UTC m=+481.928798759" watchObservedRunningTime="2026-07-28 19:10:14.487463557 +0000 UTC m=+481.929384064"
Jul 28 19:10:15.360961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.360924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:15.365549 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.365522    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:15.473856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.473831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:15.474920 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.474895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:10:15.533897 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.533857    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:15.653089 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.653011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:15.653089 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.653047    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:15.653302 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.653126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.655797 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.655774    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 19:10:15.693180 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.693155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.693307 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.693235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlzk\" (UniqueName: \"kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.794577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.794542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.794733 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.794615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlzk\" (UniqueName: \"kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.795174 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.795151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.803159 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.803132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlzk\" (UniqueName: \"kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk\") pod \"limitador-limitador-64c94fd64f-6fx69\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:15.963937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:15.963865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:16.115198 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.115169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:16.116614 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:16.116578    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1de0cc_6573_4961_b52f_ecc20bcf804d.slice/crio-9ddce2ffc2461c80fbd8b95a7d21b92ce8be833dccdfaad2b75dd1f2ad911431 WatchSource:0}: Error finding container 9ddce2ffc2461c80fbd8b95a7d21b92ce8be833dccdfaad2b75dd1f2ad911431: Status 404 returned error can't find the container with id 9ddce2ffc2461c80fbd8b95a7d21b92ce8be833dccdfaad2b75dd1f2ad911431
Jul 28 19:10:16.386188 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.386157    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:16.438270 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.438239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:16.438419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.438337    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:16.440840 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.440822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r8kdh\""
Jul 28 19:10:16.480780 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.480746    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" event={"ID":"ba1de0cc-6573-4961-b52f-ecc20bcf804d","Type":"ContainerStarted","Data":"9ddce2ffc2461c80fbd8b95a7d21b92ce8be833dccdfaad2b75dd1f2ad911431"}
Jul 28 19:10:16.501367 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.501345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5rm\" (UniqueName: \"kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm\") pod \"authorino-58cd955675-7sbts\" (UID: \"e24c3c33-01a1-425f-8055-38bdf5019399\") " pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:16.602038 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.602009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5rm\" (UniqueName: \"kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm\") pod \"authorino-58cd955675-7sbts\" (UID: \"e24c3c33-01a1-425f-8055-38bdf5019399\") " pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:16.611257 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.611204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5rm\" (UniqueName: \"kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm\") pod \"authorino-58cd955675-7sbts\" (UID: \"e24c3c33-01a1-425f-8055-38bdf5019399\") " pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:16.747876 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.747811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:16.909838 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:16.909644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:16.912543 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:16.912516    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode24c3c33_01a1_425f_8055_38bdf5019399.slice/crio-2a3a0eabf8cd5412bc96967b26e14e6c5d769308fe0646b6b257b8f3fd94f4b5 WatchSource:0}: Error finding container 2a3a0eabf8cd5412bc96967b26e14e6c5d769308fe0646b6b257b8f3fd94f4b5: Status 404 returned error can't find the container with id 2a3a0eabf8cd5412bc96967b26e14e6c5d769308fe0646b6b257b8f3fd94f4b5
Jul 28 19:10:17.488707 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:17.488672    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7sbts" event={"ID":"e24c3c33-01a1-425f-8055-38bdf5019399","Type":"ContainerStarted","Data":"2a3a0eabf8cd5412bc96967b26e14e6c5d769308fe0646b6b257b8f3fd94f4b5"}
Jul 28 19:10:20.805379 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:20.805346    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:22.531552 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.531509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7sbts" event={"ID":"e24c3c33-01a1-425f-8055-38bdf5019399","Type":"ContainerStarted","Data":"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"}
Jul 28 19:10:22.532028 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.531557    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7sbts" podUID="e24c3c33-01a1-425f-8055-38bdf5019399" containerName="authorino" containerID="cri-o://77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e" gracePeriod=30
Jul 28 19:10:22.533324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.533302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" event={"ID":"ba1de0cc-6573-4961-b52f-ecc20bcf804d","Type":"ContainerStarted","Data":"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"}
Jul 28 19:10:22.533468 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.533401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:22.546608 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.546568    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7sbts" podStartSLOduration=1.807764621 podStartE2EDuration="6.54655566s" totalImagesPullingTime="4.738791039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:22.544755077 +0000 UTC m=+489.986675588" watchObservedRunningTime="2026-07-28 19:10:22.54655566 +0000 UTC m=+489.988476168"
Jul 28 19:10:22.562709 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.562672    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" podStartSLOduration=2.040123287 podStartE2EDuration="7.562661519s" totalImagesPullingTime="5.522538232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:22.560701073 +0000 UTC m=+490.002621582" watchObservedRunningTime="2026-07-28 19:10:22.562661519 +0000 UTC m=+490.004582026"
Jul 28 19:10:22.776459 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.776429    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:22.859917 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.859893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z5rm\" (UniqueName: \"kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm\") pod \"e24c3c33-01a1-425f-8055-38bdf5019399\" (UID: \"e24c3c33-01a1-425f-8055-38bdf5019399\") "
Jul 28 19:10:22.861795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.861773    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm" (OuterVolumeSpecName: "kube-api-access-6z5rm") pod "e24c3c33-01a1-425f-8055-38bdf5019399" (UID: "e24c3c33-01a1-425f-8055-38bdf5019399"). InnerVolumeSpecName "kube-api-access-6z5rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:22.961130 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:22.961104    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z5rm\" (UniqueName: \"kubernetes.io/projected/e24c3c33-01a1-425f-8055-38bdf5019399-kube-api-access-6z5rm\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:10:23.539678 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.539643    2577 generic.go:356] "Generic (PLEG): container finished" podID="e24c3c33-01a1-425f-8055-38bdf5019399" containerID="77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e" exitCode=0
Jul 28 19:10:23.540108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.539685    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7sbts"
Jul 28 19:10:23.540108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.539730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7sbts" event={"ID":"e24c3c33-01a1-425f-8055-38bdf5019399","Type":"ContainerDied","Data":"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"}
Jul 28 19:10:23.540108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.539768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7sbts" event={"ID":"e24c3c33-01a1-425f-8055-38bdf5019399","Type":"ContainerDied","Data":"2a3a0eabf8cd5412bc96967b26e14e6c5d769308fe0646b6b257b8f3fd94f4b5"}
Jul 28 19:10:23.540108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.539785    2577 scope.go:117] "RemoveContainer" containerID="77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"
Jul 28 19:10:23.548981 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.548964    2577 scope.go:117] "RemoveContainer" containerID="77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"
Jul 28 19:10:23.549260 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:10:23.549205    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e\": container with ID starting with 77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e not found: ID does not exist" containerID="77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"
Jul 28 19:10:23.549314 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.549274    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e"} err="failed to get container status \"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e\": rpc error: code = NotFound desc = could not find container \"77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e\": container with ID starting with 77c8bedc18c6ad54b656b6158e4399004fd5c35c151c7c2c6d1421a350623c7e not found: ID does not exist"
Jul 28 19:10:23.558716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.558697    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:23.563169 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:23.563151    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7sbts"]
Jul 28 19:10:25.052368 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:25.052329    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e24c3c33-01a1-425f-8055-38bdf5019399" path="/var/lib/kubelet/pods/e24c3c33-01a1-425f-8055-38bdf5019399/volumes"
Jul 28 19:10:30.215128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.215094    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:30.215559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.215269    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" podUID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" containerName="limitador" containerID="cri-o://ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c" gracePeriod=30
Jul 28 19:10:30.217738 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.217715    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:30.782270 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.778398    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:30.826840 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.826812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nlzk\" (UniqueName: \"kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk\") pod \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") "
Jul 28 19:10:30.826992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.826886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file\") pod \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\" (UID: \"ba1de0cc-6573-4961-b52f-ecc20bcf804d\") "
Jul 28 19:10:30.827209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.827185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file" (OuterVolumeSpecName: "config-file") pod "ba1de0cc-6573-4961-b52f-ecc20bcf804d" (UID: "ba1de0cc-6573-4961-b52f-ecc20bcf804d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:10:30.828773 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.828754    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk" (OuterVolumeSpecName: "kube-api-access-2nlzk") pod "ba1de0cc-6573-4961-b52f-ecc20bcf804d" (UID: "ba1de0cc-6573-4961-b52f-ecc20bcf804d"). InnerVolumeSpecName "kube-api-access-2nlzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:30.927881 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.927839    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nlzk\" (UniqueName: \"kubernetes.io/projected/ba1de0cc-6573-4961-b52f-ecc20bcf804d-kube-api-access-2nlzk\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:10:30.927881 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:30.927879    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba1de0cc-6573-4961-b52f-ecc20bcf804d-config-file\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:10:31.586625 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.586589    2577 generic.go:356] "Generic (PLEG): container finished" podID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" containerID="ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c" exitCode=0
Jul 28 19:10:31.587031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.586639    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69"
Jul 28 19:10:31.587031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.586667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" event={"ID":"ba1de0cc-6573-4961-b52f-ecc20bcf804d","Type":"ContainerDied","Data":"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"}
Jul 28 19:10:31.587031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.586717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fx69" event={"ID":"ba1de0cc-6573-4961-b52f-ecc20bcf804d","Type":"ContainerDied","Data":"9ddce2ffc2461c80fbd8b95a7d21b92ce8be833dccdfaad2b75dd1f2ad911431"}
Jul 28 19:10:31.587031 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.586742    2577 scope.go:117] "RemoveContainer" containerID="ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"
Jul 28 19:10:31.597054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.597031    2577 scope.go:117] "RemoveContainer" containerID="ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"
Jul 28 19:10:31.597362 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:10:31.597342    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c\": container with ID starting with ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c not found: ID does not exist" containerID="ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"
Jul 28 19:10:31.597419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.597372    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c"} err="failed to get container status \"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c\": rpc error: code = NotFound desc = could not find container \"ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c\": container with ID starting with ece396209449cafeb8becec08461cff94d97a1457f047db63eef514c166c250c not found: ID does not exist"
Jul 28 19:10:31.615257 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.615234    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:31.625232 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:31.623318    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fx69"]
Jul 28 19:10:32.123797 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.123765    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"]
Jul 28 19:10:32.124205 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124191    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" containerName="limitador"
Jul 28 19:10:32.124269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124205    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" containerName="limitador"
Jul 28 19:10:32.124269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124231    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24c3c33-01a1-425f-8055-38bdf5019399" containerName="authorino"
Jul 28 19:10:32.124269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124240    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24c3c33-01a1-425f-8055-38bdf5019399" containerName="authorino"
Jul 28 19:10:32.124376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124344    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" containerName="limitador"
Jul 28 19:10:32.124376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.124358    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e24c3c33-01a1-425f-8055-38bdf5019399" containerName="authorino"
Jul 28 19:10:32.128728 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.128710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.131493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.131432    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:10:32.131493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.131432    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:10:32.131493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.131435    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 19:10:32.132314 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.132278    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8phbc\""
Jul 28 19:10:32.134919 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.134902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"]
Jul 28 19:10:32.239414 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.239383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5fefb82-012d-4973-81ad-ce250f42e34e-data\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.239578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.239458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdd2l\" (UniqueName: \"kubernetes.io/projected/f5fefb82-012d-4973-81ad-ce250f42e34e-kube-api-access-xdd2l\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.340446 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.340417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdd2l\" (UniqueName: \"kubernetes.io/projected/f5fefb82-012d-4973-81ad-ce250f42e34e-kube-api-access-xdd2l\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.340591 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.340472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5fefb82-012d-4973-81ad-ce250f42e34e-data\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.340795 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.340781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5fefb82-012d-4973-81ad-ce250f42e34e-data\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.349253 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.349231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdd2l\" (UniqueName: \"kubernetes.io/projected/f5fefb82-012d-4973-81ad-ce250f42e34e-kube-api-access-xdd2l\") pod \"postgres-69fb4594cf-m55rt\" (UID: \"f5fefb82-012d-4973-81ad-ce250f42e34e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.439990 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.439927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:32.569884 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.569863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"]
Jul 28 19:10:32.571527 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:32.571504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fefb82_012d_4973_81ad_ce250f42e34e.slice/crio-74e4df5d82df4ce89b308be42630ed0de18998bf5aa2fc0b51f74de7b1c1aa92 WatchSource:0}: Error finding container 74e4df5d82df4ce89b308be42630ed0de18998bf5aa2fc0b51f74de7b1c1aa92: Status 404 returned error can't find the container with id 74e4df5d82df4ce89b308be42630ed0de18998bf5aa2fc0b51f74de7b1c1aa92
Jul 28 19:10:32.593773 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:32.593739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt" event={"ID":"f5fefb82-012d-4973-81ad-ce250f42e34e","Type":"ContainerStarted","Data":"74e4df5d82df4ce89b308be42630ed0de18998bf5aa2fc0b51f74de7b1c1aa92"}
Jul 28 19:10:33.053550 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:33.053520    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba1de0cc-6573-4961-b52f-ecc20bcf804d" path="/var/lib/kubelet/pods/ba1de0cc-6573-4961-b52f-ecc20bcf804d/volumes"
Jul 28 19:10:37.631020 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:37.630985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt" event={"ID":"f5fefb82-012d-4973-81ad-ce250f42e34e","Type":"ContainerStarted","Data":"0cdb347671ade2913a0cac466f7c4c8927befca7d9438dea8a9753a12dc2371a"}
Jul 28 19:10:37.631513 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:37.631105    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:37.645999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:37.645960    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt" podStartSLOduration=1.008799341 podStartE2EDuration="5.645946093s" totalImagesPullingTime="4.637146752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:37.645702002 +0000 UTC m=+505.087622512" watchObservedRunningTime="2026-07-28 19:10:37.645946093 +0000 UTC m=+505.087866603"
Jul 28 19:10:43.670231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:43.670177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m55rt"
Jul 28 19:10:44.181246 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.181195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6lnhj"]
Jul 28 19:10:44.188128 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.188096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.190784 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.190761    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r8kdh\""
Jul 28 19:10:44.194571 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.194546    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6lnhj"]
Jul 28 19:10:44.348891 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.348857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrxch\" (UniqueName: \"kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch\") pod \"authorino-5d4b67c446-6lnhj\" (UID: \"30c1a3d4-1c64-437d-847f-ca6c4eb3d164\") " pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.383827 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.383795    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6lnhj"]
Jul 28 19:10:44.384078 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:10:44.384058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xrxch], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-6lnhj" podUID="30c1a3d4-1c64-437d-847f-ca6c4eb3d164"
Jul 28 19:10:44.449380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.449275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrxch\" (UniqueName: \"kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch\") pod \"authorino-5d4b67c446-6lnhj\" (UID: \"30c1a3d4-1c64-437d-847f-ca6c4eb3d164\") " pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.461229 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.461184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrxch\" (UniqueName: \"kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch\") pod \"authorino-5d4b67c446-6lnhj\" (UID: \"30c1a3d4-1c64-437d-847f-ca6c4eb3d164\") " pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.470577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.470548    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:10:44.475496 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.475478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.478022 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.477987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 19:10:44.480316 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.480290    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:10:44.550654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.550619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrkbv\" (UniqueName: \"kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.550832 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.550733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.651776 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.651743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.651937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.651789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrkbv\" (UniqueName: \"kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.654313 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.654288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.659388 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.659365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrkbv\" (UniqueName: \"kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv\") pod \"authorino-8576dc96ff-h4zf8\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") " pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.677162 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.677134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.685348 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.685329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:44.786006 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.785916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:10:44.853884 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.853845    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrxch\" (UniqueName: \"kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch\") pod \"30c1a3d4-1c64-437d-847f-ca6c4eb3d164\" (UID: \"30c1a3d4-1c64-437d-847f-ca6c4eb3d164\") "
Jul 28 19:10:44.855905 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.855883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch" (OuterVolumeSpecName: "kube-api-access-xrxch") pod "30c1a3d4-1c64-437d-847f-ca6c4eb3d164" (UID: "30c1a3d4-1c64-437d-847f-ca6c4eb3d164"). InnerVolumeSpecName "kube-api-access-xrxch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:44.918406 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.918382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:10:44.920895 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:44.920870    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a32c941_a6ff_4771_972e_27d60b76369b.slice/crio-b2c86b85e106fd78c449aabd9b1312dd8ef43e5b4ed2ef079910e570c64c3a5e WatchSource:0}: Error finding container b2c86b85e106fd78c449aabd9b1312dd8ef43e5b4ed2ef079910e570c64c3a5e: Status 404 returned error can't find the container with id b2c86b85e106fd78c449aabd9b1312dd8ef43e5b4ed2ef079910e570c64c3a5e
Jul 28 19:10:44.955314 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:44.955291    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrxch\" (UniqueName: \"kubernetes.io/projected/30c1a3d4-1c64-437d-847f-ca6c4eb3d164-kube-api-access-xrxch\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:10:45.684560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.684458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" event={"ID":"9a32c941-a6ff-4771-972e-27d60b76369b","Type":"ContainerStarted","Data":"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"}
Jul 28 19:10:45.684560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.684509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" event={"ID":"9a32c941-a6ff-4771-972e-27d60b76369b","Type":"ContainerStarted","Data":"b2c86b85e106fd78c449aabd9b1312dd8ef43e5b4ed2ef079910e570c64c3a5e"}
Jul 28 19:10:45.684560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.684474    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6lnhj"
Jul 28 19:10:45.704835 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.704787    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" podStartSLOduration=1.240325039 podStartE2EDuration="1.7047729s" totalImagesPullingTime="464.447861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:45.70211374 +0000 UTC m=+513.144034254" watchObservedRunningTime="2026-07-28 19:10:45.7047729 +0000 UTC m=+513.146693415"
Jul 28 19:10:45.734123 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.734086    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6lnhj"]
Jul 28 19:10:45.743560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:45.743533    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6lnhj"]
Jul 28 19:10:47.052724 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:47.052686    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30c1a3d4-1c64-437d-847f-ca6c4eb3d164" path="/var/lib/kubelet/pods/30c1a3d4-1c64-437d-847f-ca6c4eb3d164/volumes"
Jul 28 19:10:50.306225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.306182    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:10:50.382827 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.382792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:10:50.382980 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.382893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.388039 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.388004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:10:50.388200 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.388041    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5k8mx\""
Jul 28 19:10:50.388200 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.388041    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:10:50.500642 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.500608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dfqj\" (UniqueName: \"kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.500819 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.500753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.502998 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.502979    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:10:50.542354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.542328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:10:50.542479 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.542414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.601714 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.601691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.601851 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.601741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dfqj\" (UniqueName: \"kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.604065 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.604043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.610713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.610691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dfqj\" (UniqueName: \"kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj\") pod \"maas-controller-768b58bcc5-qfs2j\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") " pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.693262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.693235    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:50.702584 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.702559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.702698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.702614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r26f\" (UniqueName: \"kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.804136 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.804092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r26f\" (UniqueName: \"kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.804310 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.804235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.807086 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.807059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.813327 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.813302    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r26f\" (UniqueName: \"kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f\") pod \"maas-controller-67b7b484f5-b8tp9\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") " pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.851946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.851803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:50.852815 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:50.852790    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:10:50.854267 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:50.854247    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f6b9b1_992e_4159_b5ca_c9bbcdc33fde.slice/crio-d0b40c7f64f9782d560615486eeef3f7432765de1c3e6d2300bce5a43fd87b13 WatchSource:0}: Error finding container d0b40c7f64f9782d560615486eeef3f7432765de1c3e6d2300bce5a43fd87b13: Status 404 returned error can't find the container with id d0b40c7f64f9782d560615486eeef3f7432765de1c3e6d2300bce5a43fd87b13
Jul 28 19:10:51.024909 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:51.024881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:10:51.026705 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:10:51.026675    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef0edf69_c290_41a1_8226_3a60c2ce25ea.slice/crio-e5e041a5c143531a3385901f52cb6cfcaa8b41ad3f0706098cb2f9ca64005d22 WatchSource:0}: Error finding container e5e041a5c143531a3385901f52cb6cfcaa8b41ad3f0706098cb2f9ca64005d22: Status 404 returned error can't find the container with id e5e041a5c143531a3385901f52cb6cfcaa8b41ad3f0706098cb2f9ca64005d22
Jul 28 19:10:51.725695 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:51.725656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" event={"ID":"ef0edf69-c290-41a1-8226-3a60c2ce25ea","Type":"ContainerStarted","Data":"e5e041a5c143531a3385901f52cb6cfcaa8b41ad3f0706098cb2f9ca64005d22"}
Jul 28 19:10:51.727241 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:51.727190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" event={"ID":"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde","Type":"ContainerStarted","Data":"d0b40c7f64f9782d560615486eeef3f7432765de1c3e6d2300bce5a43fd87b13"}
Jul 28 19:10:53.740940 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.740907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" event={"ID":"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde","Type":"ContainerStarted","Data":"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"}
Jul 28 19:10:53.741441 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.741098    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:10:53.742737 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.742697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" event={"ID":"ef0edf69-c290-41a1-8226-3a60c2ce25ea","Type":"ContainerStarted","Data":"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"}
Jul 28 19:10:53.742878 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.742750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:10:53.758413 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.758368    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" podStartSLOduration=0.964085952 podStartE2EDuration="3.758354242s" totalImagesPullingTime="2.79426829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:53.756447066 +0000 UTC m=+521.198367587" watchObservedRunningTime="2026-07-28 19:10:53.758354242 +0000 UTC m=+521.200274749"
Jul 28 19:10:53.776785 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:10:53.776742    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" podStartSLOduration=1.161859183 podStartE2EDuration="3.776729256s" totalImagesPullingTime="2.614870073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:53.775303951 +0000 UTC m=+521.217224476" watchObservedRunningTime="2026-07-28 19:10:53.776729256 +0000 UTC m=+521.218649764"
Jul 28 19:11:04.751629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:04.751600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:11:04.752090 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:04.751669    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:11:04.810620 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:04.810589    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:11:04.810828 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:04.810802    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" podUID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" containerName="manager" containerID="cri-o://09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28" gracePeriod=30
Jul 28 19:11:05.064611 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.064588    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:11:05.126713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.126688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert\") pod \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") "
Jul 28 19:11:05.126849 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.126754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dfqj\" (UniqueName: \"kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj\") pod \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\" (UID: \"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde\") "
Jul 28 19:11:05.128933 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.128907    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" (UID: "68f6b9b1-992e-4159-b5ca-c9bbcdc33fde"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:11:05.128933 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.128917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj" (OuterVolumeSpecName: "kube-api-access-9dfqj") pod "68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" (UID: "68f6b9b1-992e-4159-b5ca-c9bbcdc33fde"). InnerVolumeSpecName "kube-api-access-9dfqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:11:05.228488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.228435    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-webhook-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:11:05.228488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.228458    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dfqj\" (UniqueName: \"kubernetes.io/projected/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde-kube-api-access-9dfqj\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:11:05.816686 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.816651    2577 generic.go:356] "Generic (PLEG): container finished" podID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" containerID="09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28" exitCode=0
Jul 28 19:11:05.817110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.816695    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768b58bcc5-qfs2j"
Jul 28 19:11:05.817110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.816698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" event={"ID":"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde","Type":"ContainerDied","Data":"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"}
Jul 28 19:11:05.817110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.816756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768b58bcc5-qfs2j" event={"ID":"68f6b9b1-992e-4159-b5ca-c9bbcdc33fde","Type":"ContainerDied","Data":"d0b40c7f64f9782d560615486eeef3f7432765de1c3e6d2300bce5a43fd87b13"}
Jul 28 19:11:05.817110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.816778    2577 scope.go:117] "RemoveContainer" containerID="09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"
Jul 28 19:11:05.826770 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.826754    2577 scope.go:117] "RemoveContainer" containerID="09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"
Jul 28 19:11:05.827067 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:11:05.827051    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28\": container with ID starting with 09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28 not found: ID does not exist" containerID="09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"
Jul 28 19:11:05.827106 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.827083    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28"} err="failed to get container status \"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28\": rpc error: code = NotFound desc = could not find container \"09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28\": container with ID starting with 09b4ef63c8e072ed9e6356f3577fcaaaa41960b82fda5026e951f0a24ab34e28 not found: ID does not exist"
Jul 28 19:11:05.840392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.840368    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:11:05.846534 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:05.846514    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-768b58bcc5-qfs2j"]
Jul 28 19:11:07.052147 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:07.052116    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" path="/var/lib/kubelet/pods/68f6b9b1-992e-4159-b5ca-c9bbcdc33fde/volumes"
Jul 28 19:11:25.642858 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.642821    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"]
Jul 28 19:11:25.643469 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.643454    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" containerName="manager"
Jul 28 19:11:25.643521 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.643473    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" containerName="manager"
Jul 28 19:11:25.643590 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.643580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f6b9b1-992e-4159-b5ca-c9bbcdc33fde" containerName="manager"
Jul 28 19:11:25.650863 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.650845    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.656794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.656758    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 19:11:25.658578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.658546    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:11:25.661335 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.661307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"]
Jul 28 19:11:25.698409 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.698374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-maas-api-tls\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.698547 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.698418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-openshift-service-ca\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.698547 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.698493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxk9f\" (UniqueName: \"kubernetes.io/projected/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-kube-api-access-kxk9f\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.799124 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.799093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-maas-api-tls\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.799124 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.799126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-openshift-service-ca\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.799386 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.799163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxk9f\" (UniqueName: \"kubernetes.io/projected/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-kube-api-access-kxk9f\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.799743 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.799721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-openshift-service-ca\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.801603 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.801579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-maas-api-tls\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.823951 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.823930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxk9f\" (UniqueName: \"kubernetes.io/projected/e9193b8c-3b87-4ddb-9818-36ecf1a322aa-kube-api-access-kxk9f\") pod \"maas-api-fbf6787c5-5qpj6\" (UID: \"e9193b8c-3b87-4ddb-9818-36ecf1a322aa\") " pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:25.962973 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:25.962905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:26.109783 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.109757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"]
Jul 28 19:11:26.111109 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:11:26.111081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9193b8c_3b87_4ddb_9818_36ecf1a322aa.slice/crio-eb9805ac2d49f8633d0d1f3341a948872a2785defec689643c9001aeaaa2be61 WatchSource:0}: Error finding container eb9805ac2d49f8633d0d1f3341a948872a2785defec689643c9001aeaaa2be61: Status 404 returned error can't find the container with id eb9805ac2d49f8633d0d1f3341a948872a2785defec689643c9001aeaaa2be61
Jul 28 19:11:26.451815 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.451787    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"]
Jul 28 19:11:26.456846 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.456825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.460209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.460190    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 19:11:26.475925 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.475901    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"]
Jul 28 19:11:26.506718 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.506690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74defc7f-a7f4-4c68-8609-a51b37bd58cf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.506839 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.506743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88g8l\" (UniqueName: \"kubernetes.io/projected/74defc7f-a7f4-4c68-8609-a51b37bd58cf-kube-api-access-88g8l\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.608016 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.607974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74defc7f-a7f4-4c68-8609-a51b37bd58cf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.608170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.608047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88g8l\" (UniqueName: \"kubernetes.io/projected/74defc7f-a7f4-4c68-8609-a51b37bd58cf-kube-api-access-88g8l\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.608717 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.608699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74defc7f-a7f4-4c68-8609-a51b37bd58cf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.625406 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.625376    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88g8l\" (UniqueName: \"kubernetes.io/projected/74defc7f-a7f4-4c68-8609-a51b37bd58cf-kube-api-access-88g8l\") pod \"payload-pre-processing-687dbb6cf6-8sd84\" (UID: \"74defc7f-a7f4-4c68-8609-a51b37bd58cf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.768202 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.768117    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:26.916914 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.916892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"]
Jul 28 19:11:26.918765 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:11:26.918727    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74defc7f_a7f4_4c68_8609_a51b37bd58cf.slice/crio-8ea0dcd353142ac5f5367ce7bfcb5f4daa8843a7b0a784eefa9514aa29f26536 WatchSource:0}: Error finding container 8ea0dcd353142ac5f5367ce7bfcb5f4daa8843a7b0a784eefa9514aa29f26536: Status 404 returned error can't find the container with id 8ea0dcd353142ac5f5367ce7bfcb5f4daa8843a7b0a784eefa9514aa29f26536
Jul 28 19:11:26.940520 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.940496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84" event={"ID":"74defc7f-a7f4-4c68-8609-a51b37bd58cf","Type":"ContainerStarted","Data":"8ea0dcd353142ac5f5367ce7bfcb5f4daa8843a7b0a784eefa9514aa29f26536"}
Jul 28 19:11:26.941670 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:26.941649    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6" event={"ID":"e9193b8c-3b87-4ddb-9818-36ecf1a322aa","Type":"ContainerStarted","Data":"eb9805ac2d49f8633d0d1f3341a948872a2785defec689643c9001aeaaa2be61"}
Jul 28 19:11:28.959132 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:28.959086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6" event={"ID":"e9193b8c-3b87-4ddb-9818-36ecf1a322aa","Type":"ContainerStarted","Data":"37be0351a380d7097f192e9ed382c901872ac6a341dbdc1bc63ff05d3cf4dcf7"}
Jul 28 19:11:28.959539 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:28.959249    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:29.003236 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:28.992623    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6" podStartSLOduration=1.69245944 podStartE2EDuration="3.992603296s" totalImagesPullingTime="2.300143856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:11:28.990575865 +0000 UTC m=+556.432496373" watchObservedRunningTime="2026-07-28 19:11:28.992603296 +0000 UTC m=+556.434523805"
Jul 28 19:11:29.966978 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:29.966939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84" event={"ID":"74defc7f-a7f4-4c68-8609-a51b37bd58cf","Type":"ContainerStarted","Data":"d8ba04926544b536c0b735411b31ce9e55f689774139edf83d227eedda079268"}
Jul 28 19:11:29.967414 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:29.967101    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:34.972820 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:34.972792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-fbf6787c5-5qpj6"
Jul 28 19:11:34.993945 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:34.993899    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84" podStartSLOduration=6.904191009 podStartE2EDuration="8.993884851s" totalImagesPullingTime="2.089693842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:11:29.987833024 +0000 UTC m=+557.429753533" watchObservedRunningTime="2026-07-28 19:11:34.993884851 +0000 UTC m=+562.435805359"
Jul 28 19:11:35.003178 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.003151    2577 generic.go:356] "Generic (PLEG): container finished" podID="61d287db-9f2e-4137-9552-18b78a231d2b" containerID="a3481f9e0518b5d25b80f4dc3c8fe4c0ae7a5648c3b2ea90f6327b73379c9085" exitCode=0
Jul 28 19:11:35.003338 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.003243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" event={"ID":"61d287db-9f2e-4137-9552-18b78a231d2b","Type":"ContainerDied","Data":"a3481f9e0518b5d25b80f4dc3c8fe4c0ae7a5648c3b2ea90f6327b73379c9085"}
Jul 28 19:11:35.003721 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.003703    2577 scope.go:117] "RemoveContainer" containerID="a3481f9e0518b5d25b80f4dc3c8fe4c0ae7a5648c3b2ea90f6327b73379c9085"
Jul 28 19:11:35.004782 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.004756    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:11:35.004862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.004832    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:11:35.004926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:35.004866    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:11:36.012929 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:36.012888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" event={"ID":"61d287db-9f2e-4137-9552-18b78a231d2b","Type":"ContainerStarted","Data":"df9c3c7ab5f5ef5f48a20e5a85c3fafe12789c2c68c10444a1eff8da84fdede5"}
Jul 28 19:11:36.360668 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:36.360631    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:11:36.360890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:36.360868    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-lrtmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 28 19:11:36.360951 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:36.360918    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" podUID="61d287db-9f2e-4137-9552-18b78a231d2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 28 19:11:37.360629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:37.360591    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-lrtmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 28 19:11:37.361029 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:37.360647    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" podUID="61d287db-9f2e-4137-9552-18b78a231d2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 28 19:11:38.360167 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:38.360131    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-lrtmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 28 19:11:38.360372 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:38.360184    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" podUID="61d287db-9f2e-4137-9552-18b78a231d2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 28 19:11:39.360810 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:39.360777    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-lrtmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 28 19:11:39.361189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:39.360830    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn" podUID="61d287db-9f2e-4137-9552-18b78a231d2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 28 19:11:40.363933 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:40.363903    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:11:40.364363 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:40.364171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:11:40.364927 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:40.364909    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-lrtmn"
Jul 28 19:11:40.977921 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:40.977893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8sd84"
Jul 28 19:11:56.640052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.640016    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"]
Jul 28 19:11:56.644197 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.644176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.646668 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.646645    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:11:56.647555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.647535    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-twk7j\""
Jul 28 19:11:56.647555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.647552    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:11:56.647686 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.647535    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 19:11:56.655191 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.655167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"]
Jul 28 19:11:56.798866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.798734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.799149 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.799123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st8g9\" (UniqueName: \"kubernetes.io/projected/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-kube-api-access-st8g9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.799297 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.799259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.799366 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.799329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.799423 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.799360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.799423 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.799404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900528 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900528 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900528 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900778 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900778 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st8g9\" (UniqueName: \"kubernetes.io/projected/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-kube-api-access-st8g9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900778 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900986 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.900986 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.900978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.902756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.902736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.903051 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.903032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.908691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.908664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st8g9\" (UniqueName: \"kubernetes.io/projected/9e3d715c-5b16-426a-9fa1-a9b0023bdd5d-kube-api-access-st8g9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vxjmg\" (UID: \"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:56.956262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:56.956234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:11:57.091751 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:57.091725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"]
Jul 28 19:11:57.094824 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:11:57.094788    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3d715c_5b16_426a_9fa1_a9b0023bdd5d.slice/crio-20c2234844ccbf1706dd879a619c4a1afa3ab44479e00c002ba1e911c8537709 WatchSource:0}: Error finding container 20c2234844ccbf1706dd879a619c4a1afa3ab44479e00c002ba1e911c8537709: Status 404 returned error can't find the container with id 20c2234844ccbf1706dd879a619c4a1afa3ab44479e00c002ba1e911c8537709
Jul 28 19:11:57.131595 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:57.131552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg" event={"ID":"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d","Type":"ContainerStarted","Data":"20c2234844ccbf1706dd879a619c4a1afa3ab44479e00c002ba1e911c8537709"}
Jul 28 19:11:59.144345 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:59.144309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg" event={"ID":"9e3d715c-5b16-426a-9fa1-a9b0023bdd5d","Type":"ContainerStarted","Data":"58c1c4386c61e45c38aece49be1019f59d4af8d61b506b735bd59c1fd4e502dd"}
Jul 28 19:11:59.164842 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:11:59.164791    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg" podStartSLOduration=1.869024221 podStartE2EDuration="3.164776009s" totalImagesPullingTime="1.295751788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:11:59.162996769 +0000 UTC m=+586.604917278" watchObservedRunningTime="2026-07-28 19:11:59.164776009 +0000 UTC m=+586.606696517"
Jul 28 19:12:06.950962 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.950930    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"]
Jul 28 19:12:06.954441 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.954422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.956641 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.956620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:12:06.956746 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.956672    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:12:06.957182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.957158    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 19:12:06.966257 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.966235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"]
Jul 28 19:12:06.971745 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.971727    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:12:06.992138 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x96xz\" (UniqueName: \"kubernetes.io/projected/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-kube-api-access-x96xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.992265 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.992309 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.992309 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.992381 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:06.992426 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:06.992375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x96xz\" (UniqueName: \"kubernetes.io/projected/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-kube-api-access-x96xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093670 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093670 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.093670 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.093626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.095892 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.095866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.096085 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.096062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.101556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.101533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x96xz\" (UniqueName: \"kubernetes.io/projected/728f3d2c-e2c1-4f13-bc33-3759a93e8ea1-kube-api-access-x96xz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ml9hb\" (UID: \"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.200629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.200600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vxjmg"
Jul 28 19:12:07.267441 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.267366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:07.403406 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:07.403377    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"]
Jul 28 19:12:07.404788 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:12:07.404764    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod728f3d2c_e2c1_4f13_bc33_3759a93e8ea1.slice/crio-4764c32e44246e0bdd9befd62b90fafc77e1dfb20e96b944b70d7986821162cd WatchSource:0}: Error finding container 4764c32e44246e0bdd9befd62b90fafc77e1dfb20e96b944b70d7986821162cd: Status 404 returned error can't find the container with id 4764c32e44246e0bdd9befd62b90fafc77e1dfb20e96b944b70d7986821162cd
Jul 28 19:12:08.198505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:08.198472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb" event={"ID":"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1","Type":"ContainerStarted","Data":"8c3ce6559e1cd877afeb278fda38b05ed8a8091ee446438e46eff8ef2a3dd52c"}
Jul 28 19:12:08.198916 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:08.198526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb" event={"ID":"728f3d2c-e2c1-4f13-bc33-3759a93e8ea1","Type":"ContainerStarted","Data":"4764c32e44246e0bdd9befd62b90fafc77e1dfb20e96b944b70d7986821162cd"}
Jul 28 19:12:08.221353 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:08.221301    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb" podStartSLOduration=1.982150954 podStartE2EDuration="2.22128772s" totalImagesPullingTime="239.136766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:12:08.219260592 +0000 UTC m=+595.661181101" watchObservedRunningTime="2026-07-28 19:12:08.22128772 +0000 UTC m=+595.663208229"
Jul 28 19:12:17.267506 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:17.267473    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:17.268008 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:17.267538    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:17.280230 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:17.280189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:12:18.273189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:12:18.273159    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ml9hb"
Jul 28 19:13:58.790839 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:58.790802    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:13:58.791334 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:58.790981    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" podUID="9a32c941-a6ff-4771-972e-27d60b76369b" containerName="authorino" containerID="cri-o://585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a" gracePeriod=30
Jul 28 19:13:59.058443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.058416    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:13:59.193261 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.193195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrkbv\" (UniqueName: \"kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv\") pod \"9a32c941-a6ff-4771-972e-27d60b76369b\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") "
Jul 28 19:13:59.193443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.193320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert\") pod \"9a32c941-a6ff-4771-972e-27d60b76369b\" (UID: \"9a32c941-a6ff-4771-972e-27d60b76369b\") "
Jul 28 19:13:59.195332 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.195303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv" (OuterVolumeSpecName: "kube-api-access-wrkbv") pod "9a32c941-a6ff-4771-972e-27d60b76369b" (UID: "9a32c941-a6ff-4771-972e-27d60b76369b"). InnerVolumeSpecName "kube-api-access-wrkbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:13:59.204231 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.204192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9a32c941-a6ff-4771-972e-27d60b76369b" (UID: "9a32c941-a6ff-4771-972e-27d60b76369b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:13:59.294505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.294440    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrkbv\" (UniqueName: \"kubernetes.io/projected/9a32c941-a6ff-4771-972e-27d60b76369b-kube-api-access-wrkbv\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:13:59.294505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.294463    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a32c941-a6ff-4771-972e-27d60b76369b-tls-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:13:59.870196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.870158    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a32c941-a6ff-4771-972e-27d60b76369b" containerID="585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a" exitCode=0
Jul 28 19:13:59.870659 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.870197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8576dc96ff-h4zf8"
Jul 28 19:13:59.870659 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.870255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" event={"ID":"9a32c941-a6ff-4771-972e-27d60b76369b","Type":"ContainerDied","Data":"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"}
Jul 28 19:13:59.870659 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.870300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8576dc96ff-h4zf8" event={"ID":"9a32c941-a6ff-4771-972e-27d60b76369b","Type":"ContainerDied","Data":"b2c86b85e106fd78c449aabd9b1312dd8ef43e5b4ed2ef079910e570c64c3a5e"}
Jul 28 19:13:59.870659 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.870317    2577 scope.go:117] "RemoveContainer" containerID="585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"
Jul 28 19:13:59.880109 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.880094    2577 scope.go:117] "RemoveContainer" containerID="585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"
Jul 28 19:13:59.880373 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:13:59.880341    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a\": container with ID starting with 585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a not found: ID does not exist" containerID="585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"
Jul 28 19:13:59.880473 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.880381    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a"} err="failed to get container status \"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a\": rpc error: code = NotFound desc = could not find container \"585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a\": container with ID starting with 585b2ebc0874c9654790cdb884f3ca8a12b8b9c99106dc5e0c5c6536b8e1627a not found: ID does not exist"
Jul 28 19:13:59.891462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.891437    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:13:59.895863 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:13:59.895844    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8576dc96ff-h4zf8"]
Jul 28 19:14:01.053713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:01.053679    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a32c941-a6ff-4771-972e-27d60b76369b" path="/var/lib/kubelet/pods/9a32c941-a6ff-4771-972e-27d60b76369b/volumes"
Jul 28 19:14:46.098810 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.098774    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:14:46.099509 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.099011    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" podUID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" containerName="manager" containerID="cri-o://a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46" gracePeriod=30
Jul 28 19:14:46.363243 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.363164    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:14:46.505180 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.505146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert\") pod \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") "
Jul 28 19:14:46.505375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.505259    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r26f\" (UniqueName: \"kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f\") pod \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\" (UID: \"ef0edf69-c290-41a1-8226-3a60c2ce25ea\") "
Jul 28 19:14:46.507241 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.507195    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ef0edf69-c290-41a1-8226-3a60c2ce25ea" (UID: "ef0edf69-c290-41a1-8226-3a60c2ce25ea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:14:46.507241 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.507199    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f" (OuterVolumeSpecName: "kube-api-access-4r26f") pod "ef0edf69-c290-41a1-8226-3a60c2ce25ea" (UID: "ef0edf69-c290-41a1-8226-3a60c2ce25ea"). InnerVolumeSpecName "kube-api-access-4r26f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:14:46.605988 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.605962    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef0edf69-c290-41a1-8226-3a60c2ce25ea-webhook-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:14:46.605988 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:46.605983    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r26f\" (UniqueName: \"kubernetes.io/projected/ef0edf69-c290-41a1-8226-3a60c2ce25ea-kube-api-access-4r26f\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:14:47.145349 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.145315    2577 generic.go:356] "Generic (PLEG): container finished" podID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" containerID="a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46" exitCode=0
Jul 28 19:14:47.145757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.145347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" event={"ID":"ef0edf69-c290-41a1-8226-3a60c2ce25ea","Type":"ContainerDied","Data":"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"}
Jul 28 19:14:47.145757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.145372    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-b8tp9"
Jul 28 19:14:47.145757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.145422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-b8tp9" event={"ID":"ef0edf69-c290-41a1-8226-3a60c2ce25ea","Type":"ContainerDied","Data":"e5e041a5c143531a3385901f52cb6cfcaa8b41ad3f0706098cb2f9ca64005d22"}
Jul 28 19:14:47.145757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.145444    2577 scope.go:117] "RemoveContainer" containerID="a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"
Jul 28 19:14:47.154429 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.154411    2577 scope.go:117] "RemoveContainer" containerID="a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"
Jul 28 19:14:47.154685 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:14:47.154666    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46\": container with ID starting with a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46 not found: ID does not exist" containerID="a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"
Jul 28 19:14:47.154752 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.154693    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46"} err="failed to get container status \"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46\": rpc error: code = NotFound desc = could not find container \"a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46\": container with ID starting with a9f18d85286a14437d67413865d3316bfb5745e688818653ef09622004ba5e46 not found: ID does not exist"
Jul 28 19:14:47.169813 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.169791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:14:47.177225 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:47.177197    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-b8tp9"]
Jul 28 19:14:49.052481 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:14:49.052450    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" path="/var/lib/kubelet/pods/ef0edf69-c290-41a1-8226-3a60c2ce25ea/volumes"
Jul 28 19:15:00.163907 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.163877    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"]
Jul 28 19:15:00.164346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164318    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" containerName="manager"
Jul 28 19:15:00.164346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164330    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" containerName="manager"
Jul 28 19:15:00.164420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164357    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a32c941-a6ff-4771-972e-27d60b76369b" containerName="authorino"
Jul 28 19:15:00.164420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164366    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a32c941-a6ff-4771-972e-27d60b76369b" containerName="authorino"
Jul 28 19:15:00.164494 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164428    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef0edf69-c290-41a1-8226-3a60c2ce25ea" containerName="manager"
Jul 28 19:15:00.164494 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.164439    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a32c941-a6ff-4771-972e-27d60b76369b" containerName="authorino"
Jul 28 19:15:00.168995 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.168978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"
Jul 28 19:15:00.171961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.171940    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:15:00.178452 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.178435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"
Jul 28 19:15:00.253855 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.253825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"]
Jul 28 19:15:00.513690 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.513664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"]
Jul 28 19:15:00.515276 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:15:00.515239    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5dd639_cbef_49c4_ae5f_c5e67a728148.slice/crio-e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef WatchSource:0}: Error finding container e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef: Status 404 returned error can't find the container with id e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef
Jul 28 19:15:00.517185 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:00.517170    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:15:01.229032 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:01.228993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j" event={"ID":"1b5dd639-cbef-49c4-ae5f-c5e67a728148","Type":"ContainerStarted","Data":"e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef"}
Jul 28 19:15:04.249586 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:04.249546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j" event={"ID":"1b5dd639-cbef-49c4-ae5f-c5e67a728148","Type":"ContainerStarted","Data":"39fd0a09859ece62cbe7cae3553cbc53280ae02d8798f4051ff8fce4498018e2"}
Jul 28 19:15:04.269441 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:04.269396    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j" podStartSLOduration=0.942277841 podStartE2EDuration="4.269383111s" totalImagesPullingTime="3.32710527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:15:04.266733167 +0000 UTC m=+771.708653674" watchObservedRunningTime="2026-07-28 19:15:04.269383111 +0000 UTC m=+771.711303618"
Jul 28 19:15:05.257476 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:05.257443    2577 generic.go:356] "Generic (PLEG): container finished" podID="1b5dd639-cbef-49c4-ae5f-c5e67a728148" containerID="39fd0a09859ece62cbe7cae3553cbc53280ae02d8798f4051ff8fce4498018e2" exitCode=0
Jul 28 19:15:05.257871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:05.257523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j" event={"ID":"1b5dd639-cbef-49c4-ae5f-c5e67a728148","Type":"ContainerDied","Data":"39fd0a09859ece62cbe7cae3553cbc53280ae02d8798f4051ff8fce4498018e2"}
Jul 28 19:15:06.395824 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:06.395801    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"
Jul 28 19:15:07.271312 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:07.271284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j"
Jul 28 19:15:07.271312 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:07.271301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754435-6wt6j" event={"ID":"1b5dd639-cbef-49c4-ae5f-c5e67a728148","Type":"ContainerDied","Data":"e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef"}
Jul 28 19:15:07.271504 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:07.271343    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e73b73b33c54e7026c4dac8406146108ef88290c736dcd905ca05fef1333ccef"
Jul 28 19:15:48.003376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.003288    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:15:48.003957 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.003942    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b5dd639-cbef-49c4-ae5f-c5e67a728148" containerName="cleanup"
Jul 28 19:15:48.004033 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.003962    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b5dd639-cbef-49c4-ae5f-c5e67a728148" containerName="cleanup"
Jul 28 19:15:48.004109 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.004093    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b5dd639-cbef-49c4-ae5f-c5e67a728148" containerName="cleanup"
Jul 28 19:15:48.007550 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.007522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.011805 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.011781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:15:48.011938 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.011840    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5k8mx\""
Jul 28 19:15:48.012321 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.012288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:15:48.016392 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.016370    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:15:48.054701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.054672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prfkd\" (UniqueName: \"kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.054853 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.054833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.155539 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.155504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.155707 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.155557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prfkd\" (UniqueName: \"kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.157904 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.157886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.171373 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.171347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prfkd\" (UniqueName: \"kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd\") pod \"maas-controller-67b7b484f5-t4ngt\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") " pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.320476 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.320391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:48.461859 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.461833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:15:48.463434 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:15:48.463404    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d456733_5b70_4c00_b465_6f056388b4e0.slice/crio-d36840fef3be7d605768e853a65cfc80f513966ab96899783a30de75a1b76f12 WatchSource:0}: Error finding container d36840fef3be7d605768e853a65cfc80f513966ab96899783a30de75a1b76f12: Status 404 returned error can't find the container with id d36840fef3be7d605768e853a65cfc80f513966ab96899783a30de75a1b76f12
Jul 28 19:15:48.507346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:48.507314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" event={"ID":"5d456733-5b70-4c00-b465-6f056388b4e0","Type":"ContainerStarted","Data":"d36840fef3be7d605768e853a65cfc80f513966ab96899783a30de75a1b76f12"}
Jul 28 19:15:49.514290 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:49.514257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" event={"ID":"5d456733-5b70-4c00-b465-6f056388b4e0","Type":"ContainerStarted","Data":"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"}
Jul 28 19:15:49.514700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:49.514318    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:15:49.536864 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:15:49.536818    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" podStartSLOduration=2.035747757 podStartE2EDuration="2.53680474s" totalImagesPullingTime="501.056983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:15:49.534709717 +0000 UTC m=+816.976630224" watchObservedRunningTime="2026-07-28 19:15:49.53680474 +0000 UTC m=+816.978725248"
Jul 28 19:16:00.525387 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:00.525357    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:16:05.161154 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.161102    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:16:05.161628 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.161347    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" podUID="5d456733-5b70-4c00-b465-6f056388b4e0" containerName="manager" containerID="cri-o://1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0" gracePeriod=30
Jul 28 19:16:05.411614 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.411554    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:16:05.507311 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.507266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prfkd\" (UniqueName: \"kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd\") pod \"5d456733-5b70-4c00-b465-6f056388b4e0\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") "
Jul 28 19:16:05.507490 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.507404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert\") pod \"5d456733-5b70-4c00-b465-6f056388b4e0\" (UID: \"5d456733-5b70-4c00-b465-6f056388b4e0\") "
Jul 28 19:16:05.509378 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.509348    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5d456733-5b70-4c00-b465-6f056388b4e0" (UID: "5d456733-5b70-4c00-b465-6f056388b4e0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:16:05.509493 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.509418    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd" (OuterVolumeSpecName: "kube-api-access-prfkd") pod "5d456733-5b70-4c00-b465-6f056388b4e0" (UID: "5d456733-5b70-4c00-b465-6f056388b4e0"). InnerVolumeSpecName "kube-api-access-prfkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:16:05.608324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.608290    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d456733-5b70-4c00-b465-6f056388b4e0-webhook-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:16:05.608324 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.608323    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prfkd\" (UniqueName: \"kubernetes.io/projected/5d456733-5b70-4c00-b465-6f056388b4e0-kube-api-access-prfkd\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:16:05.611858 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.611829    2577 generic.go:356] "Generic (PLEG): container finished" podID="5d456733-5b70-4c00-b465-6f056388b4e0" containerID="1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0" exitCode=0
Jul 28 19:16:05.611996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.611885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-t4ngt"
Jul 28 19:16:05.611996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.611918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" event={"ID":"5d456733-5b70-4c00-b465-6f056388b4e0","Type":"ContainerDied","Data":"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"}
Jul 28 19:16:05.611996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.611972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-t4ngt" event={"ID":"5d456733-5b70-4c00-b465-6f056388b4e0","Type":"ContainerDied","Data":"d36840fef3be7d605768e853a65cfc80f513966ab96899783a30de75a1b76f12"}
Jul 28 19:16:05.611996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.611994    2577 scope.go:117] "RemoveContainer" containerID="1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"
Jul 28 19:16:05.621910 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.621891    2577 scope.go:117] "RemoveContainer" containerID="1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"
Jul 28 19:16:05.622149 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:16:05.622131    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0\": container with ID starting with 1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0 not found: ID does not exist" containerID="1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"
Jul 28 19:16:05.622199 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.622157    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0"} err="failed to get container status \"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0\": rpc error: code = NotFound desc = could not find container \"1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0\": container with ID starting with 1552062b22c2e1a5ae13ddd0057471dab1809d69ed8f727f40381253f4f525a0 not found: ID does not exist"
Jul 28 19:16:05.638344 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.638320    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:16:05.646732 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:05.646707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-t4ngt"]
Jul 28 19:16:06.425757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.425720    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b7b484f5-fnkst"]
Jul 28 19:16:06.426163 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.426151    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d456733-5b70-4c00-b465-6f056388b4e0" containerName="manager"
Jul 28 19:16:06.426163 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.426164    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d456733-5b70-4c00-b465-6f056388b4e0" containerName="manager"
Jul 28 19:16:06.426266 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.426248    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d456733-5b70-4c00-b465-6f056388b4e0" containerName="manager"
Jul 28 19:16:06.430667 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.430648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.433535 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.433511    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:16:06.433645 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.433615    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:16:06.434444 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.434427    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5k8mx\""
Jul 28 19:16:06.441416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.441394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-fnkst"]
Jul 28 19:16:06.516443 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.516410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45ckg\" (UniqueName: \"kubernetes.io/projected/fac12459-d1ae-4b7f-a88d-5c078997df36-kube-api-access-45ckg\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.516609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.516452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fac12459-d1ae-4b7f-a88d-5c078997df36-webhook-cert\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.616882 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.616849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fac12459-d1ae-4b7f-a88d-5c078997df36-webhook-cert\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.617030 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.616983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45ckg\" (UniqueName: \"kubernetes.io/projected/fac12459-d1ae-4b7f-a88d-5c078997df36-kube-api-access-45ckg\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.619588 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.619562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fac12459-d1ae-4b7f-a88d-5c078997df36-webhook-cert\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.628668 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.628643    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45ckg\" (UniqueName: \"kubernetes.io/projected/fac12459-d1ae-4b7f-a88d-5c078997df36-kube-api-access-45ckg\") pod \"maas-controller-67b7b484f5-fnkst\" (UID: \"fac12459-d1ae-4b7f-a88d-5c078997df36\") " pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.741486 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.741395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:06.880696 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:06.880657    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b7b484f5-fnkst"]
Jul 28 19:16:06.884599 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:16:06.884572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac12459_d1ae_4b7f_a88d_5c078997df36.slice/crio-25ee9d7c3f6ee656a6acd643170fb4ff0de40402d18ca44214d049de68635085 WatchSource:0}: Error finding container 25ee9d7c3f6ee656a6acd643170fb4ff0de40402d18ca44214d049de68635085: Status 404 returned error can't find the container with id 25ee9d7c3f6ee656a6acd643170fb4ff0de40402d18ca44214d049de68635085
Jul 28 19:16:07.053383 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:07.053305    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d456733-5b70-4c00-b465-6f056388b4e0" path="/var/lib/kubelet/pods/5d456733-5b70-4c00-b465-6f056388b4e0/volumes"
Jul 28 19:16:07.633550 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:07.633517    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-fnkst" event={"ID":"fac12459-d1ae-4b7f-a88d-5c078997df36","Type":"ContainerStarted","Data":"be78469d4117e1bbbb0b83f401bacea0fb450b22d98a8abf125d54c63414874e"}
Jul 28 19:16:07.633924 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:07.633591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b7b484f5-fnkst" event={"ID":"fac12459-d1ae-4b7f-a88d-5c078997df36","Type":"ContainerStarted","Data":"25ee9d7c3f6ee656a6acd643170fb4ff0de40402d18ca44214d049de68635085"}
Jul 28 19:16:07.633924 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:07.633627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:16:07.665057 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:07.665002    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b7b484f5-fnkst" podStartSLOduration=1.335467517 podStartE2EDuration="1.66498708s" totalImagesPullingTime="329.519563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:16:07.664051789 +0000 UTC m=+835.105972289" watchObservedRunningTime="2026-07-28 19:16:07.66498708 +0000 UTC m=+835.106907589"
Jul 28 19:16:18.642364 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:16:18.642329    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b7b484f5-fnkst"
Jul 28 19:22:12.419756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.419723    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"]
Jul 28 19:22:12.423282 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.423267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.426584 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.426564    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6s92b\""
Jul 28 19:22:12.440885 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.440851    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"]
Jul 28 19:22:12.538728 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.538691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8z6z\" (UniqueName: \"kubernetes.io/projected/941ce64b-85c6-48ff-a585-45887464ef72-kube-api-access-w8z6z\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.538852 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.538746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/941ce64b-85c6-48ff-a585-45887464ef72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.639496 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.639467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8z6z\" (UniqueName: \"kubernetes.io/projected/941ce64b-85c6-48ff-a585-45887464ef72-kube-api-access-w8z6z\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.639616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.639519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/941ce64b-85c6-48ff-a585-45887464ef72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.639900 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.639880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/941ce64b-85c6-48ff-a585-45887464ef72-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.650365 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.650339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8z6z\" (UniqueName: \"kubernetes.io/projected/941ce64b-85c6-48ff-a585-45887464ef72-kube-api-access-w8z6z\") pod \"kuadrant-operator-controller-manager-86878c769f-8k9gc\" (UID: \"941ce64b-85c6-48ff-a585-45887464ef72\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.733305 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.733237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:12.896614 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:12.896514    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"]
Jul 28 19:22:12.899846 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:22:12.899817    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941ce64b_85c6_48ff_a585_45887464ef72.slice/crio-d0bcea49ae079a5dbe3d5880c627da3b6af92c21d186685857523a6ee257f178 WatchSource:0}: Error finding container d0bcea49ae079a5dbe3d5880c627da3b6af92c21d186685857523a6ee257f178: Status 404 returned error can't find the container with id d0bcea49ae079a5dbe3d5880c627da3b6af92c21d186685857523a6ee257f178
Jul 28 19:22:13.809311 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:13.809278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc" event={"ID":"941ce64b-85c6-48ff-a585-45887464ef72","Type":"ContainerStarted","Data":"ff6cb011a2856ab00fdb7ed50465f273dce429abc4a65410dfea3f6cbc3cc36f"}
Jul 28 19:22:13.809698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:13.809333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc" event={"ID":"941ce64b-85c6-48ff-a585-45887464ef72","Type":"ContainerStarted","Data":"d0bcea49ae079a5dbe3d5880c627da3b6af92c21d186685857523a6ee257f178"}
Jul 28 19:22:13.809698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:13.809360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:22:13.832862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:13.832815    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc" podStartSLOduration=1.832799469 podStartE2EDuration="1.832799469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:22:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:22:13.830650661 +0000 UTC m=+1201.272571170" watchObservedRunningTime="2026-07-28 19:22:13.832799469 +0000 UTC m=+1201.274719977"
Jul 28 19:22:24.818729 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:22:24.818696    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8k9gc"
Jul 28 19:30:00.133523 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.133484    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:00.137380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.137359    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"]
Jul 28 19:30:00.137501 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.137444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"
Jul 28 19:30:00.140175 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.140145    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:30:00.141077 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.141056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"
Jul 28 19:30:00.143552 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.143526    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:00.147587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.147533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"]
Jul 28 19:30:00.153437 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.153414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"
Jul 28 19:30:00.159840 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.159823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"
Jul 28 19:30:00.312488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.312455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:00.313248 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:30:00.313202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ab0e13_422c_4cf0_8e16_a0dbb9e5688f.slice/crio-d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091 WatchSource:0}: Error finding container d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091: Status 404 returned error can't find the container with id d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091
Jul 28 19:30:00.342801 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.342778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"]
Jul 28 19:30:00.343539 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:30:00.343517    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9116549b_9e55_420a_8f36_f908aa2a99ce.slice/crio-34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5 WatchSource:0}: Error finding container 34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5: Status 404 returned error can't find the container with id 34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5
Jul 28 19:30:00.539258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.539201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7" event={"ID":"9116549b-9e55-420a-8f36-f908aa2a99ce","Type":"ContainerStarted","Data":"34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5"}
Jul 28 19:30:00.540430 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:00.540404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerStarted","Data":"d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091"}
Jul 28 19:30:01.548078 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:01.547995    2577 generic.go:356] "Generic (PLEG): container finished" podID="9116549b-9e55-420a-8f36-f908aa2a99ce" containerID="23d393822afa2a0deb1b913ebd4285ffc257b8bec66c22f9d83558ac617cb19e" exitCode=0
Jul 28 19:30:01.548542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:01.548083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7" event={"ID":"9116549b-9e55-420a-8f36-f908aa2a99ce","Type":"ContainerDied","Data":"23d393822afa2a0deb1b913ebd4285ffc257b8bec66c22f9d83558ac617cb19e"}
Jul 28 19:30:01.549698 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:01.549678    2577 generic.go:356] "Generic (PLEG): container finished" podID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerID="4f3632359333614df745d16e669782a82fe1aa5ad38443d0dc407422f4106c88" exitCode=7
Jul 28 19:30:01.549800 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:01.549733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerDied","Data":"4f3632359333614df745d16e669782a82fe1aa5ad38443d0dc407422f4106c88"}
Jul 28 19:30:01.549893 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:01.549877    2577 scope.go:117] "RemoveContainer" containerID="4f3632359333614df745d16e669782a82fe1aa5ad38443d0dc407422f4106c88"
Jul 28 19:30:02.557044 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:02.556952    2577 generic.go:356] "Generic (PLEG): container finished" podID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerID="c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e" exitCode=7
Jul 28 19:30:02.557512 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:02.557035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerDied","Data":"c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e"}
Jul 28 19:30:02.557512 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:02.557093    2577 scope.go:117] "RemoveContainer" containerID="4f3632359333614df745d16e669782a82fe1aa5ad38443d0dc407422f4106c88"
Jul 28 19:30:02.557512 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:02.557298    2577 scope.go:117] "RemoveContainer" containerID="c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e"
Jul 28 19:30:02.557907 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:30:02.557559    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6_odh-ai-gateway-infra(29ab0e13-422c-4cf0-8e16-a0dbb9e5688f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f"
Jul 28 19:30:02.704202 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:02.704178    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"
Jul 28 19:30:03.563819 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:03.563789    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7"
Jul 28 19:30:03.564301 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:03.563816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754450-8k8k7" event={"ID":"9116549b-9e55-420a-8f36-f908aa2a99ce","Type":"ContainerDied","Data":"34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5"}
Jul 28 19:30:03.564301 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:03.563903    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34b7d982df4ca1daa393856467fd0eed3077beb8dc573cea70dfe649513de6b5"
Jul 28 19:30:03.565612 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:03.565592    2577 scope.go:117] "RemoveContainer" containerID="c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e"
Jul 28 19:30:03.565845 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:30:03.565819    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6_odh-ai-gateway-infra(29ab0e13-422c-4cf0-8e16-a0dbb9e5688f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f"
Jul 28 19:30:16.046781 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:16.046747    2577 scope.go:117] "RemoveContainer" containerID="c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e"
Jul 28 19:30:16.643248 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:16.643198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerStarted","Data":"726e2604a6566fbfb338ff8fbe0d92ac642d0f2c6b1b55a3f0a2ba6252014d54"}
Jul 28 19:30:16.658753 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:16.658705    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" podStartSLOduration=16.658691375 podStartE2EDuration="16.658691375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:16.656152491 +0000 UTC m=+1684.098073021" watchObservedRunningTime="2026-07-28 19:30:16.658691375 +0000 UTC m=+1684.100611883"
Jul 28 19:30:17.651488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:17.651453    2577 generic.go:356] "Generic (PLEG): container finished" podID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerID="726e2604a6566fbfb338ff8fbe0d92ac642d0f2c6b1b55a3f0a2ba6252014d54" exitCode=0
Jul 28 19:30:17.651951 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:17.651524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerDied","Data":"726e2604a6566fbfb338ff8fbe0d92ac642d0f2c6b1b55a3f0a2ba6252014d54"}
Jul 28 19:30:17.651951 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:17.651583    2577 scope.go:117] "RemoveContainer" containerID="c70a03d4d21a4fbe643ebe0565ef3435bbfc40bc066df7b40ba567f894b2971e"
Jul 28 19:30:17.666185 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:17.666165    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:18.806197 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:18.806174    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"
Jul 28 19:30:19.666932 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:19.666899    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"
Jul 28 19:30:19.667122 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:19.666939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6" event={"ID":"29ab0e13-422c-4cf0-8e16-a0dbb9e5688f","Type":"ContainerDied","Data":"d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091"}
Jul 28 19:30:19.667122 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:19.666995    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d562886cb53045fabb2798f2bfdec71a3508fa3f3eeffd9fef89bcf46a853091"
Jul 28 19:30:19.733082 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:19.733052    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:19.748166 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:19.748139    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-16c9dd30-29754450-rmvh6"]
Jul 28 19:30:21.052369 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:30:21.052334    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" path="/var/lib/kubelet/pods/29ab0e13-422c-4cf0-8e16-a0dbb9e5688f/volumes"
Jul 28 19:37:13.199609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:37:13.199536    2577 scope.go:117] "RemoveContainer" containerID="726e2604a6566fbfb338ff8fbe0d92ac642d0f2c6b1b55a3f0a2ba6252014d54"
Jul 28 19:41:23.368671 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.368635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"]
Jul 28 19:41:23.369104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369074    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369085    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369114    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9116549b-9e55-420a-8f36-f908aa2a99ce" containerName="cleanup"
Jul 28 19:41:23.369177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369119    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9116549b-9e55-420a-8f36-f908aa2a99ce" containerName="cleanup"
Jul 28 19:41:23.369177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369125    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369177 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369130    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369198    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369204    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.369322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369224    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9116549b-9e55-420a-8f36-f908aa2a99ce" containerName="cleanup"
Jul 28 19:41:23.369322 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.369231    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:23.372396 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.372380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:23.374419 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.374400    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:41:23.380283 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.380254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"]
Jul 28 19:41:23.498038 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.498005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl\" (UID: \"38ff8aca-b0d3-4c80-8739-ad5c40bce185\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:23.599042 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.599008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl\" (UID: \"38ff8aca-b0d3-4c80-8739-ad5c40bce185\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:23.599752 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.599725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl\" (UID: \"38ff8aca-b0d3-4c80-8739-ad5c40bce185\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:23.689479 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.689405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:23.828380 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:23.828353    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"]
Jul 28 19:41:23.830635 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:41:23.830609    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ff8aca_b0d3_4c80_8739_ad5c40bce185.slice/crio-9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc WatchSource:0}: Error finding container 9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc: Status 404 returned error can't find the container with id 9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc
Jul 28 19:41:24.663943 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:24.663910    2577 generic.go:356] "Generic (PLEG): container finished" podID="38ff8aca-b0d3-4c80-8739-ad5c40bce185" containerID="7580933ef62f77869bd573543aa7a4b423062000ac5ffdaa1269eae1e6274d5f" exitCode=0
Jul 28 19:41:24.664347 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:24.663968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl" event={"ID":"38ff8aca-b0d3-4c80-8739-ad5c40bce185","Type":"ContainerDied","Data":"7580933ef62f77869bd573543aa7a4b423062000ac5ffdaa1269eae1e6274d5f"}
Jul 28 19:41:24.664347 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:24.664034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl" event={"ID":"38ff8aca-b0d3-4c80-8739-ad5c40bce185","Type":"ContainerStarted","Data":"9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc"}
Jul 28 19:41:25.806198 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:25.806172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:25.921043 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:25.920960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca\") pod \"38ff8aca-b0d3-4c80-8739-ad5c40bce185\" (UID: \"38ff8aca-b0d3-4c80-8739-ad5c40bce185\") "
Jul 28 19:41:25.921369 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:25.921341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38ff8aca-b0d3-4c80-8739-ad5c40bce185" (UID: "38ff8aca-b0d3-4c80-8739-ad5c40bce185"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:26.021676 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:26.021647    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38ff8aca-b0d3-4c80-8739-ad5c40bce185-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:41:26.677294 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:26.677264    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"
Jul 28 19:41:26.677294 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:26.677280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl" event={"ID":"38ff8aca-b0d3-4c80-8739-ad5c40bce185","Type":"ContainerDied","Data":"9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc"}
Jul 28 19:41:26.677499 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:26.677323    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9997b54ea3b4c072f22f3240b705c9aa82a50dcbd175b2558519a54855ce46cc"
Jul 28 19:41:34.338468 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:34.338428    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"]
Jul 28 19:41:34.341627 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:34.341599    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e55e7d-pt5sl"]
Jul 28 19:41:35.052902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:35.052864    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38ff8aca-b0d3-4c80-8739-ad5c40bce185" path="/var/lib/kubelet/pods/38ff8aca-b0d3-4c80-8739-ad5c40bce185/volumes"
Jul 28 19:41:41.075873 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.075839    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"]
Jul 28 19:41:41.076303 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.076289    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:41.076357 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.076304    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ab0e13-422c-4cf0-8e16-a0dbb9e5688f" containerName="cleanup"
Jul 28 19:41:41.076357 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.076326    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38ff8aca-b0d3-4c80-8739-ad5c40bce185" containerName="revoke-keys"
Jul 28 19:41:41.076357 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.076332    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38ff8aca-b0d3-4c80-8739-ad5c40bce185" containerName="revoke-keys"
Jul 28 19:41:41.076509 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.076404    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38ff8aca-b0d3-4c80-8739-ad5c40bce185" containerName="revoke-keys"
Jul 28 19:41:41.079857 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.079842    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:41.082252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.082169    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:41:41.086101 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.086063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"]
Jul 28 19:41:41.162298 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.162263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj\" (UID: \"c1daa61a-73f4-4ffc-9a4d-ae2838498489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:41.263235 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.263189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj\" (UID: \"c1daa61a-73f4-4ffc-9a4d-ae2838498489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:41.263843 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.263817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj\" (UID: \"c1daa61a-73f4-4ffc-9a4d-ae2838498489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:41.394169 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.394139    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:41.749573 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.749544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"]
Jul 28 19:41:41.751348 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:41:41.751324    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1daa61a_73f4_4ffc_9a4d_ae2838498489.slice/crio-74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248 WatchSource:0}: Error finding container 74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248: Status 404 returned error can't find the container with id 74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248
Jul 28 19:41:41.766598 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:41.766563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj" event={"ID":"c1daa61a-73f4-4ffc-9a4d-ae2838498489","Type":"ContainerStarted","Data":"74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248"}
Jul 28 19:41:42.774800 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:42.774708    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1daa61a-73f4-4ffc-9a4d-ae2838498489" containerID="abca68ee42eb12d376450b3fa9133a101f2164cd163b53858dccbc88b74f18a4" exitCode=0
Jul 28 19:41:42.775162 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:42.774788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj" event={"ID":"c1daa61a-73f4-4ffc-9a4d-ae2838498489","Type":"ContainerDied","Data":"abca68ee42eb12d376450b3fa9133a101f2164cd163b53858dccbc88b74f18a4"}
Jul 28 19:41:43.918870 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:43.918844    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:43.989137 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:43.989102    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca\") pod \"c1daa61a-73f4-4ffc-9a4d-ae2838498489\" (UID: \"c1daa61a-73f4-4ffc-9a4d-ae2838498489\") "
Jul 28 19:41:43.989560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:43.989526    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1daa61a-73f4-4ffc-9a4d-ae2838498489" (UID: "c1daa61a-73f4-4ffc-9a4d-ae2838498489"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:44.090776 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:44.090742    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1daa61a-73f4-4ffc-9a4d-ae2838498489-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:41:44.787762 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:44.787733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"
Jul 28 19:41:44.787946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:44.787782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj" event={"ID":"c1daa61a-73f4-4ffc-9a4d-ae2838498489","Type":"ContainerDied","Data":"74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248"}
Jul 28 19:41:44.787946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:44.787838    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74fa9741c11d1ca3dc22b3664c637cd4712c745bf09518c2d7d637f5a86d7248"
Jul 28 19:41:45.921202 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:45.921157    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"]
Jul 28 19:41:45.923701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:45.923677    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f6911-bjxhj"]
Jul 28 19:41:47.052629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:41:47.052597    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1daa61a-73f4-4ffc-9a4d-ae2838498489" path="/var/lib/kubelet/pods/c1daa61a-73f4-4ffc-9a4d-ae2838498489/volumes"
Jul 28 19:42:52.349398 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.349315    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"]
Jul 28 19:42:52.349860 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.349790    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1daa61a-73f4-4ffc-9a4d-ae2838498489" containerName="revoke-keys"
Jul 28 19:42:52.349860 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.349801    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1daa61a-73f4-4ffc-9a4d-ae2838498489" containerName="revoke-keys"
Jul 28 19:42:52.349933 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.349875    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1daa61a-73f4-4ffc-9a4d-ae2838498489" containerName="revoke-keys"
Jul 28 19:42:52.352996 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.352980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:52.355157 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.355137    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:42:52.361880 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.361855    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"]
Jul 28 19:42:52.503764 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.503733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-29254e-hnhp2\" (UID: \"1e51be56-55ff-4632-be33-c6185207463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:52.605260 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.605139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-29254e-hnhp2\" (UID: \"1e51be56-55ff-4632-be33-c6185207463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:52.605826 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.605806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-29254e-hnhp2\" (UID: \"1e51be56-55ff-4632-be33-c6185207463c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:52.666325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.666292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:52.802117 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:52.802089    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"]
Jul 28 19:42:52.804970 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:42:52.804941    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e51be56_55ff_4632_be33_c6185207463c.slice/crio-9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406 WatchSource:0}: Error finding container 9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406: Status 404 returned error can't find the container with id 9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406
Jul 28 19:42:53.201013 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:53.200953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2" event={"ID":"1e51be56-55ff-4632-be33-c6185207463c","Type":"ContainerStarted","Data":"a36e6b01b8c93de4d93b4c60588fbee03f47028606187b551c8b1eea04bc67bd"}
Jul 28 19:42:53.201207 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:53.201022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2" event={"ID":"1e51be56-55ff-4632-be33-c6185207463c","Type":"ContainerStarted","Data":"9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406"}
Jul 28 19:42:54.214721 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:54.214686    2577 generic.go:356] "Generic (PLEG): container finished" podID="1e51be56-55ff-4632-be33-c6185207463c" containerID="a36e6b01b8c93de4d93b4c60588fbee03f47028606187b551c8b1eea04bc67bd" exitCode=0
Jul 28 19:42:54.215111 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:54.214751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2" event={"ID":"1e51be56-55ff-4632-be33-c6185207463c","Type":"ContainerDied","Data":"a36e6b01b8c93de4d93b4c60588fbee03f47028606187b551c8b1eea04bc67bd"}
Jul 28 19:42:55.361110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:55.361085    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:55.535339 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:55.535234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca\") pod \"1e51be56-55ff-4632-be33-c6185207463c\" (UID: \"1e51be56-55ff-4632-be33-c6185207463c\") "
Jul 28 19:42:55.535598 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:55.535574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e51be56-55ff-4632-be33-c6185207463c" (UID: "1e51be56-55ff-4632-be33-c6185207463c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:42:55.636431 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:55.636399    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e51be56-55ff-4632-be33-c6185207463c-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:42:56.228048 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:56.228018    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"
Jul 28 19:42:56.228273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:56.228050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2" event={"ID":"1e51be56-55ff-4632-be33-c6185207463c","Type":"ContainerDied","Data":"9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406"}
Jul 28 19:42:56.228273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:42:56.228095    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9718312dcca4d0771b5c30f8ee88c56b0c778a6f5c1fdae57c6683cff95ee406"
Jul 28 19:43:08.090992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:08.090954    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"]
Jul 28 19:43:08.093457 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:08.093432    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-29254e-hnhp2"]
Jul 28 19:43:09.052738 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:09.052703    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e51be56-55ff-4632-be33-c6185207463c" path="/var/lib/kubelet/pods/1e51be56-55ff-4632-be33-c6185207463c/volumes"
Jul 28 19:43:14.876287 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.876247    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"]
Jul 28 19:43:14.876990 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.876971    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e51be56-55ff-4632-be33-c6185207463c" containerName="revoke-keys"
Jul 28 19:43:14.877054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.876995    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e51be56-55ff-4632-be33-c6185207463c" containerName="revoke-keys"
Jul 28 19:43:14.877121 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.877109    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e51be56-55ff-4632-be33-c6185207463c" containerName="revoke-keys"
Jul 28 19:43:14.881898 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.881876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:14.884109 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.884083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"]
Jul 28 19:43:14.884390 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:14.884365    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:43:15.008551 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:15.008508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9\" (UID: \"a18c5168-1d54-417d-b267-3289c011a30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:15.109462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:15.109411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9\" (UID: \"a18c5168-1d54-417d-b267-3289c011a30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:15.110299 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:15.110263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9\" (UID: \"a18c5168-1d54-417d-b267-3289c011a30a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:15.194577 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:15.194506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:15.334175 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:15.334143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"]
Jul 28 19:43:15.335128 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:43:15.335098    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18c5168_1d54_417d_b267_3289c011a30a.slice/crio-91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d WatchSource:0}: Error finding container 91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d: Status 404 returned error can't find the container with id 91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d
Jul 28 19:43:16.348109 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:16.348024    2577 generic.go:356] "Generic (PLEG): container finished" podID="a18c5168-1d54-417d-b267-3289c011a30a" containerID="5a396258447bc38e0084def0c97417dc275190938d02850ba271ebbb9867c870" exitCode=0
Jul 28 19:43:16.348526 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:16.348107    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9" event={"ID":"a18c5168-1d54-417d-b267-3289c011a30a","Type":"ContainerDied","Data":"5a396258447bc38e0084def0c97417dc275190938d02850ba271ebbb9867c870"}
Jul 28 19:43:16.348526 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:16.348183    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9" event={"ID":"a18c5168-1d54-417d-b267-3289c011a30a","Type":"ContainerStarted","Data":"91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d"}
Jul 28 19:43:17.498165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:17.498143    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:17.533495 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:17.533466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca\") pod \"a18c5168-1d54-417d-b267-3289c011a30a\" (UID: \"a18c5168-1d54-417d-b267-3289c011a30a\") "
Jul 28 19:43:17.533807 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:17.533776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a18c5168-1d54-417d-b267-3289c011a30a" (UID: "a18c5168-1d54-417d-b267-3289c011a30a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:17.634374 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:17.634301    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18c5168-1d54-417d-b267-3289c011a30a-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:43:18.363498 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:18.363469    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"
Jul 28 19:43:18.363680 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:18.363495    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9" event={"ID":"a18c5168-1d54-417d-b267-3289c011a30a","Type":"ContainerDied","Data":"91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d"}
Jul 28 19:43:18.363680 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:18.363544    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91fbf8d939e9cde38a7c096ad5b2d6af5f1c9a5a0b59c871f3bfe8f683a6f84d"
Jul 28 19:43:30.576491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:30.576453    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"]
Jul 28 19:43:30.578987 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:30.578965    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7a6162-fg9x9"]
Jul 28 19:43:31.052983 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:43:31.052945    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18c5168-1d54-417d-b267-3289c011a30a" path="/var/lib/kubelet/pods/a18c5168-1d54-417d-b267-3289c011a30a/volumes"
Jul 28 19:44:09.637691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.637609    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:44:09.638331 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.638311    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18c5168-1d54-417d-b267-3289c011a30a" containerName="revoke-keys"
Jul 28 19:44:09.638425 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.638333    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18c5168-1d54-417d-b267-3289c011a30a" containerName="revoke-keys"
Jul 28 19:44:09.638484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.638456    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18c5168-1d54-417d-b267-3289c011a30a" containerName="revoke-keys"
Jul 28 19:44:09.641656 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.641637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.644058 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.644036    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-d66111\""
Jul 28 19:44:09.644151 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.644070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-d66111-dockercfg-j4n4l\""
Jul 28 19:44:09.650126 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.650104    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:44:09.712854 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.712823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvjwx\" (UniqueName: \"kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.713023 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.712861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.813593 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.813559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvjwx\" (UniqueName: \"kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.813760 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.813603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.814390 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.814372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.824449 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.824429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvjwx\" (UniqueName: \"kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx\") pod \"payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8\" (UID: \"37e67472-511a-4514-8797-671709717d99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:09.953365 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:09.953273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:10.101565 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:10.101536    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:44:10.103278 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:44:10.103242    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e67472_511a_4514_8797_671709717d99.slice/crio-be495ee896fee70e659e9a97babdb143934420b71a80223eb81de961cc59f215 WatchSource:0}: Error finding container be495ee896fee70e659e9a97babdb143934420b71a80223eb81de961cc59f215: Status 404 returned error can't find the container with id be495ee896fee70e659e9a97babdb143934420b71a80223eb81de961cc59f215
Jul 28 19:44:10.678041 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:10.678003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" event={"ID":"37e67472-511a-4514-8797-671709717d99","Type":"ContainerStarted","Data":"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"}
Jul 28 19:44:10.678474 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:10.678066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" event={"ID":"37e67472-511a-4514-8797-671709717d99","Type":"ContainerStarted","Data":"be495ee896fee70e659e9a97babdb143934420b71a80223eb81de961cc59f215"}
Jul 28 19:44:10.678474 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:10.678093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:44:10.701865 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:10.701810    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" podStartSLOduration=1.70179495 podStartE2EDuration="1.70179495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:10.699497523 +0000 UTC m=+2518.141418030" watchObservedRunningTime="2026-07-28 19:44:10.70179495 +0000 UTC m=+2518.143715457"
Jul 28 19:44:21.688279 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:44:21.688244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:45:00.137420 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.137380    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"]
Jul 28 19:45:00.141141 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.141123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"
Jul 28 19:45:00.145205 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.145184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:45:00.152253 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.152227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"]
Jul 28 19:45:00.154058 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.154016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"
Jul 28 19:45:00.296939 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.296908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"]
Jul 28 19:45:00.298368 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:45:00.298340    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45fc34b_c343_44c3_ab9c_d8700636f456.slice/crio-222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc WatchSource:0}: Error finding container 222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc: Status 404 returned error can't find the container with id 222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc
Jul 28 19:45:00.991192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.991148    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs" event={"ID":"d45fc34b-c343-44c3-ab9c-d8700636f456","Type":"ContainerStarted","Data":"b8081c39d11bd4b81b91a3a401d0957b8f98ec4f8293d92b8a35070f335f653d"}
Jul 28 19:45:00.991424 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:00.991207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs" event={"ID":"d45fc34b-c343-44c3-ab9c-d8700636f456","Type":"ContainerStarted","Data":"222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc"}
Jul 28 19:45:01.009832 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:01.009782    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs" podStartSLOduration=1.009768954 podStartE2EDuration="1.009768954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:01.006348787 +0000 UTC m=+2568.448269309" watchObservedRunningTime="2026-07-28 19:45:01.009768954 +0000 UTC m=+2568.451689461"
Jul 28 19:45:01.999746 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:01.999708    2577 generic.go:356] "Generic (PLEG): container finished" podID="d45fc34b-c343-44c3-ab9c-d8700636f456" containerID="b8081c39d11bd4b81b91a3a401d0957b8f98ec4f8293d92b8a35070f335f653d" exitCode=0
Jul 28 19:45:02.000161 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:01.999791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs" event={"ID":"d45fc34b-c343-44c3-ab9c-d8700636f456","Type":"ContainerDied","Data":"b8081c39d11bd4b81b91a3a401d0957b8f98ec4f8293d92b8a35070f335f653d"}
Jul 28 19:45:03.147997 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:03.147966    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"
Jul 28 19:45:04.013192 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:04.013162    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs"
Jul 28 19:45:04.013386 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:04.013208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754465-4mkrs" event={"ID":"d45fc34b-c343-44c3-ab9c-d8700636f456","Type":"ContainerDied","Data":"222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc"}
Jul 28 19:45:04.013386 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:45:04.013270    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="222e2216a684db729456cd27cd8689129c9c1e6ce8ec1a8816310352d05398bc"
Jul 28 19:48:13.315538 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:48:13.315504    2577 scope.go:117] "RemoveContainer" containerID="7580933ef62f77869bd573543aa7a4b423062000ac5ffdaa1269eae1e6274d5f"
Jul 28 19:48:13.324669 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:48:13.324652    2577 scope.go:117] "RemoveContainer" containerID="abca68ee42eb12d376450b3fa9133a101f2164cd163b53858dccbc88b74f18a4"
Jul 28 19:49:13.360165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:49:13.360086    2577 scope.go:117] "RemoveContainer" containerID="a36e6b01b8c93de4d93b4c60588fbee03f47028606187b551c8b1eea04bc67bd"
Jul 28 19:50:13.383198 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:50:13.383112    2577 scope.go:117] "RemoveContainer" containerID="5a396258447bc38e0084def0c97417dc275190938d02850ba271ebbb9867c870"
Jul 28 19:53:17.899537 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.899509    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579815693488"]
Jul 28 19:53:17.901999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.899974    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45fc34b-c343-44c3-ab9c-d8700636f456" containerName="cleanup"
Jul 28 19:53:17.901999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.899994    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45fc34b-c343-44c3-ab9c-d8700636f456" containerName="cleanup"
Jul 28 19:53:17.901999 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.900080    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45fc34b-c343-44c3-ab9c-d8700636f456" containerName="cleanup"
Jul 28 19:53:17.903166 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.903147    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:17.905896 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.905872    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pltlm\""
Jul 28 19:53:17.908949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.908926    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815693488"]
Jul 28 19:53:17.919022 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:17.918987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdgv\" (UniqueName: \"kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv\") pod \"test-curl-7503-140579815693488\" (UID: \"a233b4e8-01a5-4409-a38b-cda0c4b1916e\") " pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:18.020133 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:18.020102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdgv\" (UniqueName: \"kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv\") pod \"test-curl-7503-140579815693488\" (UID: \"a233b4e8-01a5-4409-a38b-cda0c4b1916e\") " pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:18.028059 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:18.028034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdgv\" (UniqueName: \"kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv\") pod \"test-curl-7503-140579815693488\" (UID: \"a233b4e8-01a5-4409-a38b-cda0c4b1916e\") " pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:18.216145 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:18.216073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:18.353248 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:18.353196    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815693488"]
Jul 28 19:53:18.354356 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:18.354332    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda233b4e8_01a5_4409_a38b_cda0c4b1916e.slice/crio-cd046e1346130c919f986d443c19b13d442ac8400970a51b8fbe0f024af0a876 WatchSource:0}: Error finding container cd046e1346130c919f986d443c19b13d442ac8400970a51b8fbe0f024af0a876: Status 404 returned error can't find the container with id cd046e1346130c919f986d443c19b13d442ac8400970a51b8fbe0f024af0a876
Jul 28 19:53:18.355727 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:18.355710    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:53:19.073758 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:19.073728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815693488" event={"ID":"a233b4e8-01a5-4409-a38b-cda0c4b1916e","Type":"ContainerStarted","Data":"cd046e1346130c919f986d443c19b13d442ac8400970a51b8fbe0f024af0a876"}
Jul 28 19:53:20.081345 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:20.081313    2577 generic.go:356] "Generic (PLEG): container finished" podID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" containerID="fb2d08bae1997c89f75e4772a1ae2d3ea49fd7f1b3aa6e8817f7b463b8c627a4" exitCode=0
Jul 28 19:53:20.081776 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:20.081390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815693488" event={"ID":"a233b4e8-01a5-4409-a38b-cda0c4b1916e","Type":"ContainerDied","Data":"fb2d08bae1997c89f75e4772a1ae2d3ea49fd7f1b3aa6e8817f7b463b8c627a4"}
Jul 28 19:53:21.237068 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.237045    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:21.247684 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.247665    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxdgv\" (UniqueName: \"kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv\") pod \"a233b4e8-01a5-4409-a38b-cda0c4b1916e\" (UID: \"a233b4e8-01a5-4409-a38b-cda0c4b1916e\") "
Jul 28 19:53:21.249663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.249639    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv" (OuterVolumeSpecName: "kube-api-access-bxdgv") pod "a233b4e8-01a5-4409-a38b-cda0c4b1916e" (UID: "a233b4e8-01a5-4409-a38b-cda0c4b1916e"). InnerVolumeSpecName "kube-api-access-bxdgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:21.269037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.269017    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579815693488_a233b4e8-01a5-4409-a38b-cda0c4b1916e/test-curl-7503-140579815693488/0.log"
Jul 28 19:53:21.289924 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.289897    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579815693488"]
Jul 28 19:53:21.293665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.293646    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579815693488"]
Jul 28 19:53:21.349379 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.349311    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxdgv\" (UniqueName: \"kubernetes.io/projected/a233b4e8-01a5-4409-a38b-cda0c4b1916e-kube-api-access-bxdgv\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:21.418579 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.418554    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579820736560"]
Jul 28 19:53:21.419029 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.419014    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" containerName="test-curl-7503-140579815693488"
Jul 28 19:53:21.419075 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.419030    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" containerName="test-curl-7503-140579815693488"
Jul 28 19:53:21.419117 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.419102    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" containerName="test-curl-7503-140579815693488"
Jul 28 19:53:21.422354 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.422338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:21.427946 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.427920    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820736560"]
Jul 28 19:53:21.450617 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.450583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngjz\" (UniqueName: \"kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz\") pod \"test-curl-7503-140579820736560\" (UID: \"519fdb79-7789-4d10-a375-8d6b4799fe09\") " pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:21.551926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.551897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngjz\" (UniqueName: \"kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz\") pod \"test-curl-7503-140579820736560\" (UID: \"519fdb79-7789-4d10-a375-8d6b4799fe09\") " pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:21.559767 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.559737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngjz\" (UniqueName: \"kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz\") pod \"test-curl-7503-140579820736560\" (UID: \"519fdb79-7789-4d10-a375-8d6b4799fe09\") " pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:21.733351 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.733277    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:21.866395 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:21.866370    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820736560"]
Jul 28 19:53:21.867618 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:21.867591    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod519fdb79_7789_4d10_a375_8d6b4799fe09.slice/crio-976ccdd2d0cbfe03c75b0cb11a0206b698737f3c4dfefe913bb08504a120b9d8 WatchSource:0}: Error finding container 976ccdd2d0cbfe03c75b0cb11a0206b698737f3c4dfefe913bb08504a120b9d8: Status 404 returned error can't find the container with id 976ccdd2d0cbfe03c75b0cb11a0206b698737f3c4dfefe913bb08504a120b9d8
Jul 28 19:53:22.096438 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:22.096410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820736560" event={"ID":"519fdb79-7789-4d10-a375-8d6b4799fe09","Type":"ContainerStarted","Data":"976ccdd2d0cbfe03c75b0cb11a0206b698737f3c4dfefe913bb08504a120b9d8"}
Jul 28 19:53:22.097663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:22.097645    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd046e1346130c919f986d443c19b13d442ac8400970a51b8fbe0f024af0a876"
Jul 28 19:53:22.097766 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:22.097677    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:22.108494 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:53:22.108471    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-140579815693488\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:23.050715 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:53:23.050678    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7503-140579815693488\" is forbidden: User \"system:node:ip-10-0-142-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-99.ec2.internal' and this object" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" pod="opendatahub/test-curl-7503-140579815693488"
Jul 28 19:53:23.052278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:23.052258    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a233b4e8-01a5-4409-a38b-cda0c4b1916e" path="/var/lib/kubelet/pods/a233b4e8-01a5-4409-a38b-cda0c4b1916e/volumes"
Jul 28 19:53:23.104756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:23.104729    2577 generic.go:356] "Generic (PLEG): container finished" podID="519fdb79-7789-4d10-a375-8d6b4799fe09" containerID="2d51f9d44d48d5054f1a2eb284a3a33691ffbee37070ec09cd13dbde97069fa1" exitCode=0
Jul 28 19:53:23.104890 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:23.104818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820736560" event={"ID":"519fdb79-7789-4d10-a375-8d6b4799fe09","Type":"ContainerDied","Data":"2d51f9d44d48d5054f1a2eb284a3a33691ffbee37070ec09cd13dbde97069fa1"}
Jul 28 19:53:24.252559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.252533    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:24.271867 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.271842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579820736560_519fdb79-7789-4d10-a375-8d6b4799fe09/test-curl-7503-140579820736560/0.log"
Jul 28 19:53:24.277124 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.277104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kngjz\" (UniqueName: \"kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz\") pod \"519fdb79-7789-4d10-a375-8d6b4799fe09\" (UID: \"519fdb79-7789-4d10-a375-8d6b4799fe09\") "
Jul 28 19:53:24.279048 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.279025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz" (OuterVolumeSpecName: "kube-api-access-kngjz") pod "519fdb79-7789-4d10-a375-8d6b4799fe09" (UID: "519fdb79-7789-4d10-a375-8d6b4799fe09"). InnerVolumeSpecName "kube-api-access-kngjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:24.292148 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.292125    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579820736560"]
Jul 28 19:53:24.294755 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.294734    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579820736560"]
Jul 28 19:53:24.378783 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.378718    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kngjz\" (UniqueName: \"kubernetes.io/projected/519fdb79-7789-4d10-a375-8d6b4799fe09-kube-api-access-kngjz\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:24.446769 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.446738    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:24.447199 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.447185    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519fdb79-7789-4d10-a375-8d6b4799fe09" containerName="test-curl-7503-140579820736560"
Jul 28 19:53:24.447295 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.447200    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="519fdb79-7789-4d10-a375-8d6b4799fe09" containerName="test-curl-7503-140579820736560"
Jul 28 19:53:24.447340 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.447300    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="519fdb79-7789-4d10-a375-8d6b4799fe09" containerName="test-curl-7503-140579820736560"
Jul 28 19:53:24.450806 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.450787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:24.455421 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.455396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:24.479292 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.479260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp5zw\" (UniqueName: \"kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw\") pod \"test-curl-7503-140579815794848\" (UID: \"44ee1faa-a9d3-4338-87fa-b35ab7142a5c\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:24.580733 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.580699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5zw\" (UniqueName: \"kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw\") pod \"test-curl-7503-140579815794848\" (UID: \"44ee1faa-a9d3-4338-87fa-b35ab7142a5c\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:24.588455 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.588432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp5zw\" (UniqueName: \"kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw\") pod \"test-curl-7503-140579815794848\" (UID: \"44ee1faa-a9d3-4338-87fa-b35ab7142a5c\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:24.762570 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.762494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:24.901559 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:24.901534    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:24.903069 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:24.903036    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44ee1faa_a9d3_4338_87fa_b35ab7142a5c.slice/crio-6a6aaa91a4cf5a8ba50c631f5d5e52033c9cc08ddd84b5c0246181ac1cfc5a80 WatchSource:0}: Error finding container 6a6aaa91a4cf5a8ba50c631f5d5e52033c9cc08ddd84b5c0246181ac1cfc5a80: Status 404 returned error can't find the container with id 6a6aaa91a4cf5a8ba50c631f5d5e52033c9cc08ddd84b5c0246181ac1cfc5a80
Jul 28 19:53:25.052404 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:25.052323    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="519fdb79-7789-4d10-a375-8d6b4799fe09" path="/var/lib/kubelet/pods/519fdb79-7789-4d10-a375-8d6b4799fe09/volumes"
Jul 28 19:53:25.118794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:25.118766    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820736560"
Jul 28 19:53:25.118794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:25.118794    2577 scope.go:117] "RemoveContainer" containerID="2d51f9d44d48d5054f1a2eb284a3a33691ffbee37070ec09cd13dbde97069fa1"
Jul 28 19:53:25.120309 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:25.120278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815794848" event={"ID":"44ee1faa-a9d3-4338-87fa-b35ab7142a5c","Type":"ContainerStarted","Data":"6a6aaa91a4cf5a8ba50c631f5d5e52033c9cc08ddd84b5c0246181ac1cfc5a80"}
Jul 28 19:53:26.128889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:26.128860    2577 generic.go:356] "Generic (PLEG): container finished" podID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" containerID="7a3d92c551918ccffb50e6ee6c36d725ff58226b3467aa340c56c85769c70275" exitCode=0
Jul 28 19:53:26.129276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:26.128907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815794848" event={"ID":"44ee1faa-a9d3-4338-87fa-b35ab7142a5c","Type":"ContainerDied","Data":"7a3d92c551918ccffb50e6ee6c36d725ff58226b3467aa340c56c85769c70275"}
Jul 28 19:53:27.277353 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.277332    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:27.296887 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.296835    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579815794848_44ee1faa-a9d3-4338-87fa-b35ab7142a5c/test-curl-7503-140579815794848/0.log"
Jul 28 19:53:27.307252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.307227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp5zw\" (UniqueName: \"kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw\") pod \"44ee1faa-a9d3-4338-87fa-b35ab7142a5c\" (UID: \"44ee1faa-a9d3-4338-87fa-b35ab7142a5c\") "
Jul 28 19:53:27.309295 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.309266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw" (OuterVolumeSpecName: "kube-api-access-jp5zw") pod "44ee1faa-a9d3-4338-87fa-b35ab7142a5c" (UID: "44ee1faa-a9d3-4338-87fa-b35ab7142a5c"). InnerVolumeSpecName "kube-api-access-jp5zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:27.320548 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.320524    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:27.323761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.323738    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:27.408844 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.408782    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp5zw\" (UniqueName: \"kubernetes.io/projected/44ee1faa-a9d3-4338-87fa-b35ab7142a5c-kube-api-access-jp5zw\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:27.438251 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.438226    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:27.438675 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.438663    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:27.438726 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.438679    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:27.438763 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.438755    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:27.441812 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.441798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:27.447578 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.447554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:27.509694 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.509671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq5rw\" (UniqueName: \"kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw\") pod \"test-curl-7503-140579815794848\" (UID: \"94445c72-a84b-48ee-aac3-142f276521af\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:27.611148 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.611119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq5rw\" (UniqueName: \"kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw\") pod \"test-curl-7503-140579815794848\" (UID: \"94445c72-a84b-48ee-aac3-142f276521af\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:27.618491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.618467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq5rw\" (UniqueName: \"kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw\") pod \"test-curl-7503-140579815794848\" (UID: \"94445c72-a84b-48ee-aac3-142f276521af\") " pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:27.752518 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:27.752434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:28.095606 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:28.095577    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:28.097195 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:28.097164    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94445c72_a84b_48ee_aac3_142f276521af.slice/crio-b26dcaa02d00d49631783b4b7782194a455d1c7bb91cc85b500853e1ea2d5fd9 WatchSource:0}: Error finding container b26dcaa02d00d49631783b4b7782194a455d1c7bb91cc85b500853e1ea2d5fd9: Status 404 returned error can't find the container with id b26dcaa02d00d49631783b4b7782194a455d1c7bb91cc85b500853e1ea2d5fd9
Jul 28 19:53:28.143424 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:28.143397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815794848" event={"ID":"94445c72-a84b-48ee-aac3-142f276521af","Type":"ContainerStarted","Data":"b26dcaa02d00d49631783b4b7782194a455d1c7bb91cc85b500853e1ea2d5fd9"}
Jul 28 19:53:28.144627 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:28.144609    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a6aaa91a4cf5a8ba50c631f5d5e52033c9cc08ddd84b5c0246181ac1cfc5a80"
Jul 28 19:53:28.144711 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:28.144645    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:28.174333 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:28.174291    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7503-140579815794848" oldPodUID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" podUID="94445c72-a84b-48ee-aac3-142f276521af"
Jul 28 19:53:29.052791 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:29.052757    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ee1faa-a9d3-4338-87fa-b35ab7142a5c" path="/var/lib/kubelet/pods/44ee1faa-a9d3-4338-87fa-b35ab7142a5c/volumes"
Jul 28 19:53:29.151974 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:29.151935    2577 generic.go:356] "Generic (PLEG): container finished" podID="94445c72-a84b-48ee-aac3-142f276521af" containerID="28fa35c25e1f395a7fcc7a05592a7fde65af6c78b2651423eb48835ef253ae4e" exitCode=0
Jul 28 19:53:29.152168 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:29.152015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579815794848" event={"ID":"94445c72-a84b-48ee-aac3-142f276521af","Type":"ContainerDied","Data":"28fa35c25e1f395a7fcc7a05592a7fde65af6c78b2651423eb48835ef253ae4e"}
Jul 28 19:53:30.305285 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.305262    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:30.322737 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.322711    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579815794848_94445c72-a84b-48ee-aac3-142f276521af/test-curl-7503-140579815794848/0.log"
Jul 28 19:53:30.333909 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.333882    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq5rw\" (UniqueName: \"kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw\") pod \"94445c72-a84b-48ee-aac3-142f276521af\" (UID: \"94445c72-a84b-48ee-aac3-142f276521af\") "
Jul 28 19:53:30.336054 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.336028    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw" (OuterVolumeSpecName: "kube-api-access-pq5rw") pod "94445c72-a84b-48ee-aac3-142f276521af" (UID: "94445c72-a84b-48ee-aac3-142f276521af"). InnerVolumeSpecName "kube-api-access-pq5rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:30.348191 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.348168    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:30.352348 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.352330    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579815794848"]
Jul 28 19:53:30.435403 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.435328    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq5rw\" (UniqueName: \"kubernetes.io/projected/94445c72-a84b-48ee-aac3-142f276521af-kube-api-access-pq5rw\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:30.481104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.481079    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:30.481567 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.481552    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94445c72-a84b-48ee-aac3-142f276521af" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:30.481629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.481569    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="94445c72-a84b-48ee-aac3-142f276521af" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:30.481672 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.481649    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="94445c72-a84b-48ee-aac3-142f276521af" containerName="test-curl-7503-140579815794848"
Jul 28 19:53:30.485131 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.485110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:30.493515 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.493491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:30.536566 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.536537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmj22\" (UniqueName: \"kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22\") pod \"test-curl-7503-140579820577536\" (UID: \"bb98d62d-26ad-4191-9f1d-2e10008711b3\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:30.637558 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.637528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmj22\" (UniqueName: \"kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22\") pod \"test-curl-7503-140579820577536\" (UID: \"bb98d62d-26ad-4191-9f1d-2e10008711b3\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:30.646088 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.646059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmj22\" (UniqueName: \"kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22\") pod \"test-curl-7503-140579820577536\" (UID: \"bb98d62d-26ad-4191-9f1d-2e10008711b3\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:30.797119 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.797038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:30.927267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:30.927241    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:30.928780 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:30.928752    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb98d62d_26ad_4191_9f1d_2e10008711b3.slice/crio-2e4a1b43b4be0f3d036651132d4040ad5e9026ac1f21570b261fc77f9cf069a5 WatchSource:0}: Error finding container 2e4a1b43b4be0f3d036651132d4040ad5e9026ac1f21570b261fc77f9cf069a5: Status 404 returned error can't find the container with id 2e4a1b43b4be0f3d036651132d4040ad5e9026ac1f21570b261fc77f9cf069a5
Jul 28 19:53:31.054193 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:31.054124    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94445c72-a84b-48ee-aac3-142f276521af" path="/var/lib/kubelet/pods/94445c72-a84b-48ee-aac3-142f276521af/volumes"
Jul 28 19:53:31.165655 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:31.165616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820577536" event={"ID":"bb98d62d-26ad-4191-9f1d-2e10008711b3","Type":"ContainerStarted","Data":"2e4a1b43b4be0f3d036651132d4040ad5e9026ac1f21570b261fc77f9cf069a5"}
Jul 28 19:53:31.166760 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:31.166741    2577 scope.go:117] "RemoveContainer" containerID="28fa35c25e1f395a7fcc7a05592a7fde65af6c78b2651423eb48835ef253ae4e"
Jul 28 19:53:31.166905 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:31.166775    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579815794848"
Jul 28 19:53:32.174966 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:32.174934    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb98d62d-26ad-4191-9f1d-2e10008711b3" containerID="23567fb9cfb365d04f7d2eb8f292cc628ea9bb809fdd5b6a9802ad0af2129593" exitCode=0
Jul 28 19:53:32.175336 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:32.174973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820577536" event={"ID":"bb98d62d-26ad-4191-9f1d-2e10008711b3","Type":"ContainerDied","Data":"23567fb9cfb365d04f7d2eb8f292cc628ea9bb809fdd5b6a9802ad0af2129593"}
Jul 28 19:53:33.320985 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.320960    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.338847 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.338819    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579820577536_bb98d62d-26ad-4191-9f1d-2e10008711b3/test-curl-7503-140579820577536/0.log"
Jul 28 19:53:33.362690 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.362656    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:33.364092 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.364070    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmj22\" (UniqueName: \"kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22\") pod \"bb98d62d-26ad-4191-9f1d-2e10008711b3\" (UID: \"bb98d62d-26ad-4191-9f1d-2e10008711b3\") "
Jul 28 19:53:33.366144 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.366116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22" (OuterVolumeSpecName: "kube-api-access-bmj22") pod "bb98d62d-26ad-4191-9f1d-2e10008711b3" (UID: "bb98d62d-26ad-4191-9f1d-2e10008711b3"). InnerVolumeSpecName "kube-api-access-bmj22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:33.367137 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.367114    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:33.465740 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.465676    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmj22\" (UniqueName: \"kubernetes.io/projected/bb98d62d-26ad-4191-9f1d-2e10008711b3-kube-api-access-bmj22\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:33.478783 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.478757    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:33.479237 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.479206    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb98d62d-26ad-4191-9f1d-2e10008711b3" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:33.479237 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.479233    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb98d62d-26ad-4191-9f1d-2e10008711b3" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:33.479798 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.479774    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb98d62d-26ad-4191-9f1d-2e10008711b3" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:33.485916 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.485893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.495326 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.495302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:33.566611 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.566585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kqqx\" (UniqueName: \"kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx\") pod \"test-curl-7503-140579820577536\" (UID: \"95d9ab28-5048-4e06-9536-9e629fa3c6b1\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.667475 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.667446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kqqx\" (UniqueName: \"kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx\") pod \"test-curl-7503-140579820577536\" (UID: \"95d9ab28-5048-4e06-9536-9e629fa3c6b1\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.675834 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.675814    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kqqx\" (UniqueName: \"kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx\") pod \"test-curl-7503-140579820577536\" (UID: \"95d9ab28-5048-4e06-9536-9e629fa3c6b1\") " pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.801278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.801183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:33.936288 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:33.936264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:33.937638 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:33.937611    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95d9ab28_5048_4e06_9536_9e629fa3c6b1.slice/crio-98fc05a5990ec65bcce2756fbc4627247f3d6712068243c5ec02099b36a6bc57 WatchSource:0}: Error finding container 98fc05a5990ec65bcce2756fbc4627247f3d6712068243c5ec02099b36a6bc57: Status 404 returned error can't find the container with id 98fc05a5990ec65bcce2756fbc4627247f3d6712068243c5ec02099b36a6bc57
Jul 28 19:53:34.188279 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:34.188247    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:34.188430 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:34.188249    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e4a1b43b4be0f3d036651132d4040ad5e9026ac1f21570b261fc77f9cf069a5"
Jul 28 19:53:34.189454 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:34.189433    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820577536" event={"ID":"95d9ab28-5048-4e06-9536-9e629fa3c6b1","Type":"ContainerStarted","Data":"98fc05a5990ec65bcce2756fbc4627247f3d6712068243c5ec02099b36a6bc57"}
Jul 28 19:53:34.200237 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:34.200199    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7503-140579820577536" oldPodUID="bb98d62d-26ad-4191-9f1d-2e10008711b3" podUID="95d9ab28-5048-4e06-9536-9e629fa3c6b1"
Jul 28 19:53:35.052144 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:35.052109    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb98d62d-26ad-4191-9f1d-2e10008711b3" path="/var/lib/kubelet/pods/bb98d62d-26ad-4191-9f1d-2e10008711b3/volumes"
Jul 28 19:53:35.196984 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:35.196954    2577 generic.go:356] "Generic (PLEG): container finished" podID="95d9ab28-5048-4e06-9536-9e629fa3c6b1" containerID="081eb234901109bd294baca85cc9e337c2d9c42591c6dff19a52fa1fadaa040e" exitCode=0
Jul 28 19:53:35.197137 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:35.197040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7503-140579820577536" event={"ID":"95d9ab28-5048-4e06-9536-9e629fa3c6b1","Type":"ContainerDied","Data":"081eb234901109bd294baca85cc9e337c2d9c42591c6dff19a52fa1fadaa040e"}
Jul 28 19:53:36.340739 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.340716    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:36.359200 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.359157    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7503-140579820577536_95d9ab28-5048-4e06-9536-9e629fa3c6b1/test-curl-7503-140579820577536/0.log"
Jul 28 19:53:36.380591 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.380556    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:36.384206 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.384171    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7503-140579820577536"]
Jul 28 19:53:36.394052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.394028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kqqx\" (UniqueName: \"kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx\") pod \"95d9ab28-5048-4e06-9536-9e629fa3c6b1\" (UID: \"95d9ab28-5048-4e06-9536-9e629fa3c6b1\") "
Jul 28 19:53:36.396019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.395994    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx" (OuterVolumeSpecName: "kube-api-access-5kqqx") pod "95d9ab28-5048-4e06-9536-9e629fa3c6b1" (UID: "95d9ab28-5048-4e06-9536-9e629fa3c6b1"). InnerVolumeSpecName "kube-api-access-5kqqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:36.495677 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:36.495598    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kqqx\" (UniqueName: \"kubernetes.io/projected/95d9ab28-5048-4e06-9536-9e629fa3c6b1-kube-api-access-5kqqx\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:37.052534 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:37.052495    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d9ab28-5048-4e06-9536-9e629fa3c6b1" path="/var/lib/kubelet/pods/95d9ab28-5048-4e06-9536-9e629fa3c6b1/volumes"
Jul 28 19:53:37.211602 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:37.211576    2577 scope.go:117] "RemoveContainer" containerID="081eb234901109bd294baca85cc9e337c2d9c42591c6dff19a52fa1fadaa040e"
Jul 28 19:53:37.211792 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:37.211587    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7503-140579820577536"
Jul 28 19:53:39.639652 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.639623    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:39.640108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.640094    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d9ab28-5048-4e06-9536-9e629fa3c6b1" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:39.640108 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.640109    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d9ab28-5048-4e06-9536-9e629fa3c6b1" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:39.640202 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.640194    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d9ab28-5048-4e06-9536-9e629fa3c6b1" containerName="test-curl-7503-140579820577536"
Jul 28 19:53:39.644862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.644838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:39.650673 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.650649    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:39.726639 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.726614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxwp\" (UniqueName: \"kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp\") pod \"kuadrant-operator-catalog-52vtp\" (UID: \"44a297ac-903a-4349-ae97-f3879e89c0e5\") " pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:39.829118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.829082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxwp\" (UniqueName: \"kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp\") pod \"kuadrant-operator-catalog-52vtp\" (UID: \"44a297ac-903a-4349-ae97-f3879e89c0e5\") " pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:39.837314 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.837292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxwp\" (UniqueName: \"kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp\") pod \"kuadrant-operator-catalog-52vtp\" (UID: \"44a297ac-903a-4349-ae97-f3879e89c0e5\") " pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:39.958303 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:39.958231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:40.098867 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:40.098839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:40.099934 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:53:40.099909    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a297ac_903a_4349_ae97_f3879e89c0e5.slice/crio-db67efbece519d46f7b32d304f3c3d19ee2282c8949f239dcf318f58d600883b WatchSource:0}: Error finding container db67efbece519d46f7b32d304f3c3d19ee2282c8949f239dcf318f58d600883b: Status 404 returned error can't find the container with id db67efbece519d46f7b32d304f3c3d19ee2282c8949f239dcf318f58d600883b
Jul 28 19:53:40.231263 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:40.231166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" event={"ID":"44a297ac-903a-4349-ae97-f3879e89c0e5","Type":"ContainerStarted","Data":"db67efbece519d46f7b32d304f3c3d19ee2282c8949f239dcf318f58d600883b"}
Jul 28 19:53:41.239306 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:41.239274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" event={"ID":"44a297ac-903a-4349-ae97-f3879e89c0e5","Type":"ContainerStarted","Data":"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"}
Jul 28 19:53:41.269881 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:41.269836    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" podStartSLOduration=1.9399290630000001 podStartE2EDuration="2.269820105s" totalImagesPullingTime="329.891042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:41.269649442 +0000 UTC m=+3088.711569950" watchObservedRunningTime="2026-07-28 19:53:41.269820105 +0000 UTC m=+3088.711740610"
Jul 28 19:53:49.959182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:49.959138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:49.959785 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:49.959258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:49.981651 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:49.981629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:50.321971 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:50.321896    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:50.356592 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:50.356564    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:52.312662 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:52.312625    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" podUID="44a297ac-903a-4349-ae97-f3879e89c0e5" containerName="registry-server" containerID="cri-o://19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e" gracePeriod=2
Jul 28 19:53:52.572530 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:52.572478    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:52.753716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:52.753685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdxwp\" (UniqueName: \"kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp\") pod \"44a297ac-903a-4349-ae97-f3879e89c0e5\" (UID: \"44a297ac-903a-4349-ae97-f3879e89c0e5\") "
Jul 28 19:53:52.755665 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:52.755640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp" (OuterVolumeSpecName: "kube-api-access-qdxwp") pod "44a297ac-903a-4349-ae97-f3879e89c0e5" (UID: "44a297ac-903a-4349-ae97-f3879e89c0e5"). InnerVolumeSpecName "kube-api-access-qdxwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:52.854723 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:52.854697    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdxwp\" (UniqueName: \"kubernetes.io/projected/44a297ac-903a-4349-ae97-f3879e89c0e5-kube-api-access-qdxwp\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:53:53.321011 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.320933    2577 generic.go:356] "Generic (PLEG): container finished" podID="44a297ac-903a-4349-ae97-f3879e89c0e5" containerID="19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e" exitCode=0
Jul 28 19:53:53.321011 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.320970    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-52vtp"
Jul 28 19:53:53.321011 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.320989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" event={"ID":"44a297ac-903a-4349-ae97-f3879e89c0e5","Type":"ContainerDied","Data":"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"}
Jul 28 19:53:53.321555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.321051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-52vtp" event={"ID":"44a297ac-903a-4349-ae97-f3879e89c0e5","Type":"ContainerDied","Data":"db67efbece519d46f7b32d304f3c3d19ee2282c8949f239dcf318f58d600883b"}
Jul 28 19:53:53.321555 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.321082    2577 scope.go:117] "RemoveContainer" containerID="19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"
Jul 28 19:53:53.331488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.331473    2577 scope.go:117] "RemoveContainer" containerID="19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"
Jul 28 19:53:53.331761 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:53:53.331734    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e\": container with ID starting with 19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e not found: ID does not exist" containerID="19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"
Jul 28 19:53:53.331881 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.331763    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e"} err="failed to get container status \"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e\": rpc error: code = NotFound desc = could not find container \"19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e\": container with ID starting with 19c494626b56eb0508d41692c68fa4e5811972b8dc8d7bad4bec8a295758707e not found: ID does not exist"
Jul 28 19:53:53.336973 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.336939    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:53.342492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:53.342469    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-52vtp"]
Jul 28 19:53:55.052481 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:53:55.052444    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a297ac-903a-4349-ae97-f3879e89c0e5" path="/var/lib/kubelet/pods/44a297ac-903a-4349-ae97-f3879e89c0e5/volumes"
Jul 28 19:54:06.287154 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.287117    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:54:06.287866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.287849    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a297ac-903a-4349-ae97-f3879e89c0e5" containerName="registry-server"
Jul 28 19:54:06.287917 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.287869    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a297ac-903a-4349-ae97-f3879e89c0e5" containerName="registry-server"
Jul 28 19:54:06.287987 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.287976    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a297ac-903a-4349-ae97-f3879e89c0e5" containerName="registry-server"
Jul 28 19:54:06.291824 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.291802    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.294138 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.294113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-e29fb5-openshift-default-dockercfg-8wqzs\""
Jul 28 19:54:06.300310 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.300285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:54:06.383101 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgvs\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383142    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383356 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.383700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.383412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.484977 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.484940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485170 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485376 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.485574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.485405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgvs\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.486067 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.486041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.486253 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.486059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.486535 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.486489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.486653 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.486549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.487007 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.486985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.489096 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.489075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.489416 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.489398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.501945 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.501916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.502398 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.502376    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgvs\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs\") pod \"e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") " pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.604788 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.604763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:06.772194 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.772157    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:54:06.772861 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:54:06.772838    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d2a16e_91ae_4418_8099_76a70af96957.slice/crio-f4b1978d7976659df5f456d7559a8666333529a9c18a2ccf1a0e339398c4cb2c WatchSource:0}: Error finding container f4b1978d7976659df5f456d7559a8666333529a9c18a2ccf1a0e339398c4cb2c: Status 404 returned error can't find the container with id f4b1978d7976659df5f456d7559a8666333529a9c18a2ccf1a0e339398c4cb2c
Jul 28 19:54:06.774785 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.774753    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:06.774862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.774812    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:06.774862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:06.774840    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:07.416100 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:07.416059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerStarted","Data":"c6e9915b879c7a74c46f367c9262d57443e4a340136e66316cda97d6e226b8f1"}
Jul 28 19:54:07.416508 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:07.416124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerStarted","Data":"f4b1978d7976659df5f456d7559a8666333529a9c18a2ccf1a0e339398c4cb2c"}
Jul 28 19:54:07.439680 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:07.439631    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podStartSLOduration=1.439616051 podStartE2EDuration="1.439616051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:54:07.437591912 +0000 UTC m=+3114.879512437" watchObservedRunningTime="2026-07-28 19:54:07.439616051 +0000 UTC m=+3114.881536559"
Jul 28 19:54:07.604926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:07.604894    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:07.609750 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:07.609726    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:08.422922 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:08.422893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:08.423872 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:08.423851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:11.549546 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.549512    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:54:11.553334 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.553315    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.555595 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.555573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e29fb5-dockercfg-qs47c\""
Jul 28 19:54:11.555700 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.555619    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e29fb5\""
Jul 28 19:54:11.560646 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.560624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:54:11.635353 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.635316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.635548 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.635366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5wh\" (UniqueName: \"kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.736025 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.735985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.736025 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.736030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5wh\" (UniqueName: \"kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.736705 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.736682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.744492 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.744469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5wh\" (UniqueName: \"kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh\") pod \"payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:11.865813 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:11.865783    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:12.007123 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:12.007096    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:54:12.008119 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:54:12.008095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee80e04_3eb3_4b71_aa67_25be36a3b27e.slice/crio-5460725f5093dfda929c2f863cadf9ef06500081d41f529877fa08dd6f49bcd8 WatchSource:0}: Error finding container 5460725f5093dfda929c2f863cadf9ef06500081d41f529877fa08dd6f49bcd8: Status 404 returned error can't find the container with id 5460725f5093dfda929c2f863cadf9ef06500081d41f529877fa08dd6f49bcd8
Jul 28 19:54:12.458257 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:12.458207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" event={"ID":"eee80e04-3eb3-4b71-aa67-25be36a3b27e","Type":"ContainerStarted","Data":"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"}
Jul 28 19:54:12.458467 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:12.458305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" event={"ID":"eee80e04-3eb3-4b71-aa67-25be36a3b27e","Type":"ContainerStarted","Data":"5460725f5093dfda929c2f863cadf9ef06500081d41f529877fa08dd6f49bcd8"}
Jul 28 19:54:12.458467 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:12.458364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:12.473125 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:12.473057    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" podStartSLOduration=1.4730423639999999 podStartE2EDuration="1.473042364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:54:12.472178672 +0000 UTC m=+3119.914099179" watchObservedRunningTime="2026-07-28 19:54:12.473042364 +0000 UTC m=+3119.914962873"
Jul 28 19:54:23.466414 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:23.466379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:54:27.574623 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.574591    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0d2a16e-91ae-4418-8099-76a70af96957" containerID="c6e9915b879c7a74c46f367c9262d57443e4a340136e66316cda97d6e226b8f1" exitCode=0
Jul 28 19:54:27.575004 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.574661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerDied","Data":"c6e9915b879c7a74c46f367c9262d57443e4a340136e66316cda97d6e226b8f1"}
Jul 28 19:54:27.575111 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.575097    2577 scope.go:117] "RemoveContainer" containerID="c6e9915b879c7a74c46f367c9262d57443e4a340136e66316cda97d6e226b8f1"
Jul 28 19:54:27.576158 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.576133    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:27.576262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.576201    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:27.576262 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:27.576256    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 19:54:28.583466 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:28.583422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerStarted","Data":"2608a6c69b881707a99248996a827b34d43751e061e11be2db450c7ea9bc0e80"}
Jul 28 19:54:28.605498 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:28.605458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:28.610473 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:28.610436    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Jul 28 19:54:28.610669 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:28.610502    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Jul 28 19:54:29.605488 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:29.605444    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Jul 28 19:54:29.605968 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:29.605522    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Jul 28 19:54:30.606093 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:30.605674    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Jul 28 19:54:30.606093 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:30.605745    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Jul 28 19:54:31.550273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:31.550233    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq_eee80e04-3eb3-4b71-aa67-25be36a3b27e/payload-processing/0.log"
Jul 28 19:54:31.607716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:31.607253    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Jul 28 19:54:31.607716 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:31.607371    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Jul 28 19:54:32.609124 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:32.609095    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:32.609543 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:32.609525    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:54:32.610136 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:54:32.610123    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:55:28.852056 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:28.852027    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq_eee80e04-3eb3-4b71-aa67-25be36a3b27e/payload-processing/0.log"
Jul 28 19:55:34.376629 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.376588    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:34.381397 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.381375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.384135 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.383985    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-ec8cba-serving-cert\""
Jul 28 19:55:34.388836 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.388811    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:34.443634 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.443600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.443801 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.443727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5297z\" (UniqueName: \"kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.443801 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.443766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.544961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.544528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5297z\" (UniqueName: \"kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.544961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.544629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.544961 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.544722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.545542 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.545514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.548110 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.548074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.554822 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.554798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5297z\" (UniqueName: \"kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z\") pod \"maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.699339 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.699258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:34.852292 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.852264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:34.853529 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:55:34.853501    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ee5d08_e247_4664_8376_2b31590bca67.slice/crio-35d7363f4ad2e792aee2ebd059747e871b583b63bde6201cf75612dabae31650 WatchSource:0}: Error finding container 35d7363f4ad2e792aee2ebd059747e871b583b63bde6201cf75612dabae31650: Status 404 returned error can't find the container with id 35d7363f4ad2e792aee2ebd059747e871b583b63bde6201cf75612dabae31650
Jul 28 19:55:34.993080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.992989    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:34.998045 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:34.998022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.000626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.000607    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ec8cba\""
Jul 28 19:55:35.000734 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.000709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ec8cba-dockercfg-9c7jl\""
Jul 28 19:55:35.007691 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.007659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:35.017267 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.017241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" event={"ID":"74ee5d08-e247-4664-8376-2b31590bca67","Type":"ContainerStarted","Data":"35d7363f4ad2e792aee2ebd059747e871b583b63bde6201cf75612dabae31650"}
Jul 28 19:55:35.049186 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.049160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.049342 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.049200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gp49\" (UniqueName: \"kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.150823 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.150741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.150823 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.150803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gp49\" (UniqueName: \"kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.151753 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.151727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.158569 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.158550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gp49\" (UniqueName: \"kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49\") pod \"payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.310913 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.310838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:35.448587 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:55:35.448555    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6c81f3_511a_473b_9650_4e0a53e31d01.slice/crio-bfcc53e431331630b85867469e54097ecdecee7a6cbed07fce6d8471a3c6fe12 WatchSource:0}: Error finding container bfcc53e431331630b85867469e54097ecdecee7a6cbed07fce6d8471a3c6fe12: Status 404 returned error can't find the container with id bfcc53e431331630b85867469e54097ecdecee7a6cbed07fce6d8471a3c6fe12
Jul 28 19:55:35.449436 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:35.449403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:36.026190 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.026154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" event={"ID":"74ee5d08-e247-4664-8376-2b31590bca67","Type":"ContainerStarted","Data":"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"}
Jul 28 19:55:36.026417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.026401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:36.027721 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.027697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" event={"ID":"db6c81f3-511a-473b-9650-4e0a53e31d01","Type":"ContainerStarted","Data":"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"}
Jul 28 19:55:36.027840 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.027739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" event={"ID":"db6c81f3-511a-473b-9650-4e0a53e31d01","Type":"ContainerStarted","Data":"bfcc53e431331630b85867469e54097ecdecee7a6cbed07fce6d8471a3c6fe12"}
Jul 28 19:55:36.027840 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.027769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:36.044165 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.044119    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" podStartSLOduration=1.34555483 podStartE2EDuration="2.044106498s" totalImagesPullingTime="698.551668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:36.041431564 +0000 UTC m=+3203.483352074" watchObservedRunningTime="2026-07-28 19:55:36.044106498 +0000 UTC m=+3203.486027006"
Jul 28 19:55:36.055616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:36.055569    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" podStartSLOduration=2.055555148 podStartE2EDuration="2.055555148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:36.055553015 +0000 UTC m=+3203.497473525" watchObservedRunningTime="2026-07-28 19:55:36.055555148 +0000 UTC m=+3203.497475648"
Jul 28 19:55:42.039914 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:42.039886    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:47.037076 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:47.037044    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:48.172551 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.172521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"]
Jul 28 19:55:48.176309 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.176287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:48.178434 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.178414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:55:48.184328 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.184303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"]
Jul 28 19:55:48.278295 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.278257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv\" (UID: \"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:48.379289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.379257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv\" (UID: \"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:48.379891 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.379871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv\" (UID: \"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:48.491517 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.491431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:48.631743 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:48.631714    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"]
Jul 28 19:55:48.632683 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:55:48.632642    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d92b00_1e0a_4e6f_9b16_28f2347c64e2.slice/crio-36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2 WatchSource:0}: Error finding container 36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2: Status 404 returned error can't find the container with id 36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2
Jul 28 19:55:49.124583 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:49.124543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv" event={"ID":"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2","Type":"ContainerStarted","Data":"9c63cec277d0584f1aa2a28ae2a5a336e012cf9caa856ad3b06d853ba60f6b37"}
Jul 28 19:55:49.124765 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:49.124608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv" event={"ID":"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2","Type":"ContainerStarted","Data":"36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2"}
Jul 28 19:55:49.138921 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:49.138869    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv" podStartSLOduration=1.138854791 podStartE2EDuration="1.138854791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:49.137933653 +0000 UTC m=+3216.579854162" watchObservedRunningTime="2026-07-28 19:55:49.138854791 +0000 UTC m=+3216.580775299"
Jul 28 19:55:50.132660 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:50.132620    2577 generic.go:356] "Generic (PLEG): container finished" podID="e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" containerID="9c63cec277d0584f1aa2a28ae2a5a336e012cf9caa856ad3b06d853ba60f6b37" exitCode=0
Jul 28 19:55:50.133037 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:50.132703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv" event={"ID":"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2","Type":"ContainerDied","Data":"9c63cec277d0584f1aa2a28ae2a5a336e012cf9caa856ad3b06d853ba60f6b37"}
Jul 28 19:55:51.282942 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:51.282913    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:51.411329 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:51.411252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca\") pod \"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2\" (UID: \"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2\") "
Jul 28 19:55:51.411783 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:51.411750    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" (UID: "e6d92b00-1e0a-4e6f-9b16-28f2347c64e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:55:51.512934 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:51.512900    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:52.147252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:52.147224    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"
Jul 28 19:55:52.147252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:52.147238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv" event={"ID":"e6d92b00-1e0a-4e6f-9b16-28f2347c64e2","Type":"ContainerDied","Data":"36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2"}
Jul 28 19:55:52.147462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:52.147289    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36a58b7363f73842f697d1d432fc04a760077c97d8ba19caaa8a870149bef2e2"
Jul 28 19:55:55.767258 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:55.767208    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:55.767722 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:55.767504    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" podUID="74ee5d08-e247-4664-8376-2b31590bca67" containerName="maas-api" containerID="cri-o://e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c" gracePeriod=30
Jul 28 19:55:55.830656 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:55.830615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:55.830935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:55.830881    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" podUID="db6c81f3-511a-473b-9650-4e0a53e31d01" containerName="payload-pre-processing" containerID="cri-o://43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca" gracePeriod=30
Jul 28 19:55:56.124330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.124285    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:56.128066 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.128041    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:56.128178 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.128119    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"]
Jul 28 19:55:56.130849 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.130830    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ec8cba-cbhfv"]
Jul 28 19:55:56.175643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.175596    2577 generic.go:356] "Generic (PLEG): container finished" podID="74ee5d08-e247-4664-8376-2b31590bca67" containerID="e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c" exitCode=0
Jul 28 19:55:56.175643 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.175636    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"
Jul 28 19:55:56.175862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.175663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" event={"ID":"74ee5d08-e247-4664-8376-2b31590bca67","Type":"ContainerDied","Data":"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"}
Jul 28 19:55:56.175862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.175717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc" event={"ID":"74ee5d08-e247-4664-8376-2b31590bca67","Type":"ContainerDied","Data":"35d7363f4ad2e792aee2ebd059747e871b583b63bde6201cf75612dabae31650"}
Jul 28 19:55:56.175862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.175743    2577 scope.go:117] "RemoveContainer" containerID="e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"
Jul 28 19:55:56.177252 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.177200    2577 generic.go:356] "Generic (PLEG): container finished" podID="db6c81f3-511a-473b-9650-4e0a53e31d01" containerID="43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca" exitCode=0
Jul 28 19:55:56.177417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.177256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" event={"ID":"db6c81f3-511a-473b-9650-4e0a53e31d01","Type":"ContainerDied","Data":"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"}
Jul 28 19:55:56.177417 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.177362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9" event={"ID":"db6c81f3-511a-473b-9650-4e0a53e31d01","Type":"ContainerDied","Data":"bfcc53e431331630b85867469e54097ecdecee7a6cbed07fce6d8471a3c6fe12"}
Jul 28 19:55:56.177536 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.177438    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"
Jul 28 19:55:56.188345 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.188325    2577 scope.go:117] "RemoveContainer" containerID="e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"
Jul 28 19:55:56.188655 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:55:56.188626    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c\": container with ID starting with e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c not found: ID does not exist" containerID="e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"
Jul 28 19:55:56.188756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.188656    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c"} err="failed to get container status \"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c\": rpc error: code = NotFound desc = could not find container \"e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c\": container with ID starting with e84caf12560668460ff228037847df2c11f0015d03c65dade3c01872c999ba2c not found: ID does not exist"
Jul 28 19:55:56.188756 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.188679    2577 scope.go:117] "RemoveContainer" containerID="43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"
Jul 28 19:55:56.198118 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.198099    2577 scope.go:117] "RemoveContainer" containerID="43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"
Jul 28 19:55:56.198414 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:55:56.198387    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca\": container with ID starting with 43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca not found: ID does not exist" containerID="43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"
Jul 28 19:55:56.198491 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.198420    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca"} err="failed to get container status \"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca\": rpc error: code = NotFound desc = could not find container \"43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca\": container with ID starting with 43bea167ca6a2b7422567a97b05cdac6e321ae9071bb25da0afbffbb70cda8ca not found: ID does not exist"
Jul 28 19:55:56.259462 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259436    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls\") pod \"74ee5d08-e247-4664-8376-2b31590bca67\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") "
Jul 28 19:55:56.259587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259503    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gp49\" (UniqueName: \"kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49\") pod \"db6c81f3-511a-473b-9650-4e0a53e31d01\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") "
Jul 28 19:55:56.259587 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca\") pod \"74ee5d08-e247-4664-8376-2b31590bca67\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") "
Jul 28 19:55:56.259679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5297z\" (UniqueName: \"kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z\") pod \"74ee5d08-e247-4664-8376-2b31590bca67\" (UID: \"74ee5d08-e247-4664-8376-2b31590bca67\") "
Jul 28 19:55:56.259679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume\") pod \"db6c81f3-511a-473b-9650-4e0a53e31d01\" (UID: \"db6c81f3-511a-473b-9650-4e0a53e31d01\") "
Jul 28 19:55:56.259949 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.259922    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74ee5d08-e247-4664-8376-2b31590bca67" (UID: "74ee5d08-e247-4664-8376-2b31590bca67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:55:56.260079 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.260049    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db6c81f3-511a-473b-9650-4e0a53e31d01" (UID: "db6c81f3-511a-473b-9650-4e0a53e31d01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:55:56.261641 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.261616    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49" (OuterVolumeSpecName: "kube-api-access-6gp49") pod "db6c81f3-511a-473b-9650-4e0a53e31d01" (UID: "db6c81f3-511a-473b-9650-4e0a53e31d01"). InnerVolumeSpecName "kube-api-access-6gp49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:55:56.261851 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.261831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74ee5d08-e247-4664-8376-2b31590bca67" (UID: "74ee5d08-e247-4664-8376-2b31590bca67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:55:56.261914 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.261891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z" (OuterVolumeSpecName: "kube-api-access-5297z") pod "74ee5d08-e247-4664-8376-2b31590bca67" (UID: "74ee5d08-e247-4664-8376-2b31590bca67"). InnerVolumeSpecName "kube-api-access-5297z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:55:56.360689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.360652    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee5d08-e247-4664-8376-2b31590bca67-openshift-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.360689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.360683    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5297z\" (UniqueName: \"kubernetes.io/projected/74ee5d08-e247-4664-8376-2b31590bca67-kube-api-access-5297z\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.360689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.360696    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db6c81f3-511a-473b-9650-4e0a53e31d01-plugins-config-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.360924 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.360708    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74ee5d08-e247-4664-8376-2b31590bca67-maas-api-tls\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.360924 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.360720    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gp49\" (UniqueName: \"kubernetes.io/projected/db6c81f3-511a-473b-9650-4e0a53e31d01-kube-api-access-6gp49\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.500685 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.500651    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:56.503440 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.503416    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ec8cba-69547cb556-4dwmc"]
Jul 28 19:55:56.513987 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.513964    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:56.516459 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:56.516439    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ec8cba-54656cd9fb-hfbx9"]
Jul 28 19:55:57.053772 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:57.053691    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74ee5d08-e247-4664-8376-2b31590bca67" path="/var/lib/kubelet/pods/74ee5d08-e247-4664-8376-2b31590bca67/volumes"
Jul 28 19:55:57.054136 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:57.054071    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db6c81f3-511a-473b-9650-4e0a53e31d01" path="/var/lib/kubelet/pods/db6c81f3-511a-473b-9650-4e0a53e31d01/volumes"
Jul 28 19:55:57.054386 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:55:57.054373    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" path="/var/lib/kubelet/pods/e6d92b00-1e0a-4e6f-9b16-28f2347c64e2/volumes"
Jul 28 19:56:07.463288 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463188    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"]
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463669    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" containerName="revoke-keys"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463682    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" containerName="revoke-keys"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463695    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee5d08-e247-4664-8376-2b31590bca67" containerName="maas-api"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463700    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee5d08-e247-4664-8376-2b31590bca67" containerName="maas-api"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463712    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db6c81f3-511a-473b-9650-4e0a53e31d01" containerName="payload-pre-processing"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463717    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="db6c81f3-511a-473b-9650-4e0a53e31d01" containerName="payload-pre-processing"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463788    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="74ee5d08-e247-4664-8376-2b31590bca67" containerName="maas-api"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463799    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d92b00-1e0a-4e6f-9b16-28f2347c64e2" containerName="revoke-keys"
Jul 28 19:56:07.463889 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.463807    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="db6c81f3-511a-473b-9650-4e0a53e31d01" containerName="payload-pre-processing"
Jul 28 19:56:07.468415 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.468397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:07.470975 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.470949    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:56:07.474688 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.474664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"]
Jul 28 19:56:07.569926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.569890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg\" (UID: \"ef873640-3942-459c-a57b-4d381f181c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:07.671414 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.671381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg\" (UID: \"ef873640-3942-459c-a57b-4d381f181c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:07.672014 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.671994    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg\" (UID: \"ef873640-3942-459c-a57b-4d381f181c44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:07.781068 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:07.780982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:08.129910 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:08.129883    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"]
Jul 28 19:56:08.130812 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:56:08.130785    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef873640_3942_459c_a57b_4d381f181c44.slice/crio-b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325 WatchSource:0}: Error finding container b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325: Status 404 returned error can't find the container with id b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325
Jul 28 19:56:08.259494 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:08.259456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg" event={"ID":"ef873640-3942-459c-a57b-4d381f181c44","Type":"ContainerStarted","Data":"b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325"}
Jul 28 19:56:09.268269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:09.268171    2577 generic.go:356] "Generic (PLEG): container finished" podID="ef873640-3942-459c-a57b-4d381f181c44" containerID="4d5bf10c18ddc659d7479ded012e56662c5c07e78a1c0a396914da42d01025fd" exitCode=0
Jul 28 19:56:09.268269 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:09.268224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg" event={"ID":"ef873640-3942-459c-a57b-4d381f181c44","Type":"ContainerDied","Data":"4d5bf10c18ddc659d7479ded012e56662c5c07e78a1c0a396914da42d01025fd"}
Jul 28 19:56:10.418988 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:10.418962    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:10.501693 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:10.501662    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca\") pod \"ef873640-3942-459c-a57b-4d381f181c44\" (UID: \"ef873640-3942-459c-a57b-4d381f181c44\") "
Jul 28 19:56:10.502019 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:10.501996    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef873640-3942-459c-a57b-4d381f181c44" (UID: "ef873640-3942-459c-a57b-4d381f181c44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:56:10.602822 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:10.602789    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef873640-3942-459c-a57b-4d381f181c44-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:56:11.282998 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:11.282970    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"
Jul 28 19:56:11.282998 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:11.282993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg" event={"ID":"ef873640-3942-459c-a57b-4d381f181c44","Type":"ContainerDied","Data":"b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325"}
Jul 28 19:56:11.283237 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:11.283034    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b49cf5142f4686e0ba4cb0d72c852deb302e2900673728299dd5f0474ec4c325"
Jul 28 19:56:23.719209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:23.719166    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"]
Jul 28 19:56:23.723418 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:23.723390    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf1393-rp9xg"]
Jul 28 19:56:25.053042 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:56:25.052996    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef873640-3942-459c-a57b-4d381f181c44" path="/var/lib/kubelet/pods/ef873640-3942-459c-a57b-4d381f181c44/volumes"
Jul 28 19:57:19.158189 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.158154    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:57:19.158757 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.158414    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" podUID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" containerName="payload-processing" containerID="cri-o://5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed" gracePeriod=30
Jul 28 19:57:19.435998 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.435953    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:57:19.586437 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.586404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume\") pod \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") "
Jul 28 19:57:19.586437 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.586443    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk5wh\" (UniqueName: \"kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh\") pod \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\" (UID: \"eee80e04-3eb3-4b71-aa67-25be36a3b27e\") "
Jul 28 19:57:19.586774 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.586746    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eee80e04-3eb3-4b71-aa67-25be36a3b27e" (UID: "eee80e04-3eb3-4b71-aa67-25be36a3b27e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:19.588525 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.588504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh" (OuterVolumeSpecName: "kube-api-access-bk5wh") pod "eee80e04-3eb3-4b71-aa67-25be36a3b27e" (UID: "eee80e04-3eb3-4b71-aa67-25be36a3b27e"). InnerVolumeSpecName "kube-api-access-bk5wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:19.687339 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.687255    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eee80e04-3eb3-4b71-aa67-25be36a3b27e-plugins-config-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.687339 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.687289    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk5wh\" (UniqueName: \"kubernetes.io/projected/eee80e04-3eb3-4b71-aa67-25be36a3b27e-kube-api-access-bk5wh\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.741305 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.741273    2577 generic.go:356] "Generic (PLEG): container finished" podID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" containerID="5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed" exitCode=0
Jul 28 19:57:19.741470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.741316    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"
Jul 28 19:57:19.741470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.741321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" event={"ID":"eee80e04-3eb3-4b71-aa67-25be36a3b27e","Type":"ContainerDied","Data":"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"}
Jul 28 19:57:19.741470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.741375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq" event={"ID":"eee80e04-3eb3-4b71-aa67-25be36a3b27e","Type":"ContainerDied","Data":"5460725f5093dfda929c2f863cadf9ef06500081d41f529877fa08dd6f49bcd8"}
Jul 28 19:57:19.741470 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.741404    2577 scope.go:117] "RemoveContainer" containerID="5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"
Jul 28 19:57:19.751908 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.751889    2577 scope.go:117] "RemoveContainer" containerID="5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"
Jul 28 19:57:19.752170 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:57:19.752151    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed\": container with ID starting with 5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed not found: ID does not exist" containerID="5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"
Jul 28 19:57:19.752264 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.752179    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed"} err="failed to get container status \"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed\": rpc error: code = NotFound desc = could not find container \"5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed\": container with ID starting with 5b27f2f1aa9c84d66e6b84ac1daf5a5afd5277f2cd812a80a6dd6a57bac4caed not found: ID does not exist"
Jul 28 19:57:19.764265 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.764209    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:57:19.766951 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:19.766933    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e29fb5-796585cb4b-ckjqq"]
Jul 28 19:57:21.053310 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:21.053269    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" path="/var/lib/kubelet/pods/eee80e04-3eb3-4b71-aa67-25be36a3b27e/volumes"
Jul 28 19:57:26.609023 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.608986    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:57:26.609626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.609253    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy" containerID="cri-o://2608a6c69b881707a99248996a827b34d43751e061e11be2db450c7ea9bc0e80" gracePeriod=30
Jul 28 19:57:26.986072 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.985976    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"]
Jul 28 19:57:26.986682 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986664    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef873640-3942-459c-a57b-4d381f181c44" containerName="revoke-keys"
Jul 28 19:57:26.986761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986684    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef873640-3942-459c-a57b-4d381f181c44" containerName="revoke-keys"
Jul 28 19:57:26.986761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986715    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" containerName="payload-processing"
Jul 28 19:57:26.986761 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986724    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" containerName="payload-processing"
Jul 28 19:57:26.986926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986872    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef873640-3942-459c-a57b-4d381f181c44" containerName="revoke-keys"
Jul 28 19:57:26.986926 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.986889    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee80e04-3eb3-4b71-aa67-25be36a3b27e" containerName="payload-processing"
Jul 28 19:57:26.990151 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.990129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:26.993624 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.992759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:57:26.996606 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:26.996581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"]
Jul 28 19:57:27.057148 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.057100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s\" (UID: \"3d2ae230-8f6d-4949-a307-f38c52bbd686\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:27.157931 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.157891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s\" (UID: \"3d2ae230-8f6d-4949-a307-f38c52bbd686\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:27.158568 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.158549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s\" (UID: \"3d2ae230-8f6d-4949-a307-f38c52bbd686\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:27.304346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.304262    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:27.451546 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.451517    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"]
Jul 28 19:57:27.453281 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:57:27.453246    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2ae230_8f6d_4949_a307_f38c52bbd686.slice/crio-f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d WatchSource:0}: Error finding container f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d: Status 404 returned error can't find the container with id f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d
Jul 28 19:57:27.809052 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.809019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s" event={"ID":"3d2ae230-8f6d-4949-a307-f38c52bbd686","Type":"ContainerStarted","Data":"456f1171704ff40dd038a16241403c12237c7ae6c000fbc87737ae9c14eacc53"}
Jul 28 19:57:27.809439 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.809109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s" event={"ID":"3d2ae230-8f6d-4949-a307-f38c52bbd686","Type":"ContainerStarted","Data":"f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d"}
Jul 28 19:57:27.826909 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:27.826853    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s" podStartSLOduration=1.826839858 podStartE2EDuration="1.826839858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:57:27.823498333 +0000 UTC m=+3315.265418845" watchObservedRunningTime="2026-07-28 19:57:27.826839858 +0000 UTC m=+3315.268760409"
Jul 28 19:57:28.817775 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:28.817738    2577 generic.go:356] "Generic (PLEG): container finished" podID="3d2ae230-8f6d-4949-a307-f38c52bbd686" containerID="456f1171704ff40dd038a16241403c12237c7ae6c000fbc87737ae9c14eacc53" exitCode=0
Jul 28 19:57:28.817775 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:28.817782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s" event={"ID":"3d2ae230-8f6d-4949-a307-f38c52bbd686","Type":"ContainerDied","Data":"456f1171704ff40dd038a16241403c12237c7ae6c000fbc87737ae9c14eacc53"}
Jul 28 19:57:29.965814 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:29.965793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:30.086024 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.085990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca\") pod \"3d2ae230-8f6d-4949-a307-f38c52bbd686\" (UID: \"3d2ae230-8f6d-4949-a307-f38c52bbd686\") "
Jul 28 19:57:30.086375 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.086352    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d2ae230-8f6d-4949-a307-f38c52bbd686" (UID: "3d2ae230-8f6d-4949-a307-f38c52bbd686"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:30.187589 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.187551    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ae230-8f6d-4949-a307-f38c52bbd686-maas-api-service-ca\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:30.831824 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.831792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"
Jul 28 19:57:30.831989 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.831832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s" event={"ID":"3d2ae230-8f6d-4949-a307-f38c52bbd686","Type":"ContainerDied","Data":"f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d"}
Jul 28 19:57:30.831989 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:30.831879    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4beea06440186471c60c2db4f671b7972a8d6775fdb0e163be61da6a02d1e6d"
Jul 28 19:57:31.841209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:31.841173    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0d2a16e-91ae-4418-8099-76a70af96957" containerID="2608a6c69b881707a99248996a827b34d43751e061e11be2db450c7ea9bc0e80" exitCode=0
Jul 28 19:57:31.841671 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:31.841251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerDied","Data":"2608a6c69b881707a99248996a827b34d43751e061e11be2db450c7ea9bc0e80"}
Jul 28 19:57:31.841671 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:31.841329    2577 scope.go:117] "RemoveContainer" containerID="c6e9915b879c7a74c46f367c9262d57443e4a340136e66316cda97d6e226b8f1"
Jul 28 19:57:31.885506 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:31.885158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:57:32.006068 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006001    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006084    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006109    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006270    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006304    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006361    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcgvs\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006397    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo\") pod \"d0d2a16e-91ae-4418-8099-76a70af96957\" (UID: \"d0d2a16e-91ae-4418-8099-76a70af96957\") "
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006466    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:32.006556 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006550    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006612    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006762    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-socket\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006782    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0d2a16e-91ae-4418-8099-76a70af96957-istiod-ca-cert\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006798    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-credential-socket\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006813    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-workload-certs\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.006866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.006858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:32.008907 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.008863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:32.009012 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.008960    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs" (OuterVolumeSpecName: "kube-api-access-mcgvs") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "kube-api-access-mcgvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:32.009058 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.009023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:32.009058 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.009048    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0d2a16e-91ae-4418-8099-76a70af96957" (UID: "d0d2a16e-91ae-4418-8099-76a70af96957"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:57:32.107382 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.107352    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0d2a16e-91ae-4418-8099-76a70af96957-istio-podinfo\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.107382 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.107379    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-data\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.107382 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.107388    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-istio-token\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.107702 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.107400    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0d2a16e-91ae-4418-8099-76a70af96957-istio-envoy\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.107702 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.107410    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcgvs\" (UniqueName: \"kubernetes.io/projected/d0d2a16e-91ae-4418-8099-76a70af96957-kube-api-access-mcgvs\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.202736 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.202657    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"]
Jul 28 19:57:32.206907 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.206880    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ec81e7-qdk8s"]
Jul 28 19:57:32.849273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.849244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"
Jul 28 19:57:32.849273 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.849262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv" event={"ID":"d0d2a16e-91ae-4418-8099-76a70af96957","Type":"ContainerDied","Data":"f4b1978d7976659df5f456d7559a8666333529a9c18a2ccf1a0e339398c4cb2c"}
Jul 28 19:57:32.849768 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.849343    2577 scope.go:117] "RemoveContainer" containerID="2608a6c69b881707a99248996a827b34d43751e061e11be2db450c7ea9bc0e80"
Jul 28 19:57:32.871955 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.871926    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:57:32.877887 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:32.877864    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-e29fb5-openshift-default-fbf4485f4-9mxfv"]
Jul 28 19:57:33.053303 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:33.053275    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2ae230-8f6d-4949-a307-f38c52bbd686" path="/var/lib/kubelet/pods/3d2ae230-8f6d-4949-a307-f38c52bbd686/volumes"
Jul 28 19:57:33.053713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:33.053700    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" path="/var/lib/kubelet/pods/d0d2a16e-91ae-4418-8099-76a70af96957/volumes"
Jul 28 19:57:47.350800 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.350711    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:57:47.351275 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.350946    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" podUID="37e67472-511a-4514-8797-671709717d99" containerName="payload-pre-processing" containerID="cri-o://71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394" gracePeriod=30
Jul 28 19:57:47.636547 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.636520    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:57:47.751484 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.751444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvjwx\" (UniqueName: \"kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx\") pod \"37e67472-511a-4514-8797-671709717d99\" (UID: \"37e67472-511a-4514-8797-671709717d99\") "
Jul 28 19:57:47.751679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.751514    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume\") pod \"37e67472-511a-4514-8797-671709717d99\" (UID: \"37e67472-511a-4514-8797-671709717d99\") "
Jul 28 19:57:47.751902 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.751877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37e67472-511a-4514-8797-671709717d99" (UID: "37e67472-511a-4514-8797-671709717d99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:47.753560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.753531    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx" (OuterVolumeSpecName: "kube-api-access-gvjwx") pod "37e67472-511a-4514-8797-671709717d99" (UID: "37e67472-511a-4514-8797-671709717d99"). InnerVolumeSpecName "kube-api-access-gvjwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:47.852689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.852650    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvjwx\" (UniqueName: \"kubernetes.io/projected/37e67472-511a-4514-8797-671709717d99-kube-api-access-gvjwx\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.852689 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.852683    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37e67472-511a-4514-8797-671709717d99-plugins-config-volume\") on node \"ip-10-0-142-99.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.952088 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.951998    2577 generic.go:356] "Generic (PLEG): container finished" podID="37e67472-511a-4514-8797-671709717d99" containerID="71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394" exitCode=0
Jul 28 19:57:47.952088 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.952048    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"
Jul 28 19:57:47.952346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.952075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" event={"ID":"37e67472-511a-4514-8797-671709717d99","Type":"ContainerDied","Data":"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"}
Jul 28 19:57:47.952346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.952145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8" event={"ID":"37e67472-511a-4514-8797-671709717d99","Type":"ContainerDied","Data":"be495ee896fee70e659e9a97babdb143934420b71a80223eb81de961cc59f215"}
Jul 28 19:57:47.952346 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.952170    2577 scope.go:117] "RemoveContainer" containerID="71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"
Jul 28 19:57:47.966289 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.966268    2577 scope.go:117] "RemoveContainer" containerID="71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"
Jul 28 19:57:47.966581 ip-10-0-142-99 kubenswrapper[2577]: E0728 19:57:47.966559    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394\": container with ID starting with 71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394 not found: ID does not exist" containerID="71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"
Jul 28 19:57:47.966648 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.966588    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394"} err="failed to get container status \"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394\": rpc error: code = NotFound desc = could not find container \"71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394\": container with ID starting with 71f9a235c0eb62e37fa7ceedf0ea4c22138a9428f842c6ca9b5ffb288e170394 not found: ID does not exist"
Jul 28 19:57:47.976871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.976844    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:57:47.980833 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:47.980807    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d66111-77869c56bd-rjvt8"]
Jul 28 19:57:49.060827 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:49.060786    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37e67472-511a-4514-8797-671709717d99" path="/var/lib/kubelet/pods/37e67472-511a-4514-8797-671709717d99/volumes"
Jul 28 19:57:59.969370 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:57:59.969337    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8h4zv_cc1aafe3-5f2e-4464-805c-ae1e818c7df8/manager/0.log"
Jul 28 19:58:00.086635 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:00.086596    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m9n8w_7e269572-7948-4425-8422-c70da198c1ef/manager/0.log"
Jul 28 19:58:00.209152 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:00.209114    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b7b484f5-fnkst_fac12459-d1ae-4b7f-a88d-5c078997df36/manager/0.log"
Jul 28 19:58:00.428300 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:00.428269    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dh7kc_983016fc-8d6c-4434-a5a7-47107a15cb8f/manager/0.log"
Jul 28 19:58:00.537900 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:00.537870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-fwshq_1246b0ff-4d7b-42b5-bdf3-f30f5a597482/copy-manifests/0.log"
Jul 28 19:58:00.544051 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:00.544029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-fwshq_1246b0ff-4d7b-42b5-bdf3-f30f5a597482/manager/0.log"
Jul 28 19:58:01.119654 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:01.119624    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-fbf6787c5-5qpj6_e9193b8c-3b87-4ddb-9818-36ecf1a322aa/maas-api/0.log"
Jul 28 19:58:01.223663 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:01.223633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754435-6wt6j_1b5dd639-cbef-49c4-ae5f-c5e67a728148/cleanup/0.log"
Jul 28 19:58:01.328308 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:01.328272    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754450-8k8k7_9116549b-9e55-420a-8f36-f908aa2a99ce/cleanup/0.log"
Jul 28 19:58:01.438928 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:01.438848    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754465-4mkrs_d45fc34b-c343-44c3-ab9c-d8700636f456/cleanup/0.log"
Jul 28 19:58:01.544172 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:01.544147    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-m55rt_f5fefb82-012d-4973-81ad-ce250f42e34e/postgres/0.log"
Jul 28 19:58:02.293539 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.293467    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/extract/0.log"
Jul 28 19:58:02.298507 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.298474    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/util/0.log"
Jul 28 19:58:02.304053 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.304029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/pull/0.log"
Jul 28 19:58:02.416862 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.416827    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/util/0.log"
Jul 28 19:58:02.424741 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.424719    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/pull/0.log"
Jul 28 19:58:02.436935 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.436910    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/extract/0.log"
Jul 28 19:58:02.542732 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.542691    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/util/0.log"
Jul 28 19:58:02.548713 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.548654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/pull/0.log"
Jul 28 19:58:02.554431 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.554414    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/extract/0.log"
Jul 28 19:58:02.663937 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.663908    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/util/0.log"
Jul 28 19:58:02.672867 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.672840    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/pull/0.log"
Jul 28 19:58:02.679690 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.679667    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/extract/0.log"
Jul 28 19:58:02.908650 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:02.908623    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h662z_141fde96-b7d5-4b72-8644-5b858660a319/manager/0.log"
Jul 28 19:58:03.016079 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.016048    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7v42v_31d2cef2-3c1f-46cf-ba48-b82e67777b4e/manager/0.log"
Jul 28 19:58:03.122156 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.122125    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m9j7c_a86f8dc9-b711-4d33-8a03-193f8fb5d902/kuadrant-console-plugin/0.log"
Jul 28 19:58:03.237564 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.237492    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xb75x_04b4d565-ae97-480a-98ff-f01f2724b547/registry-server/0.log"
Jul 28 19:58:03.355971 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.355926    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8k9gc_941ce64b-85c6-48ff-a585-45887464ef72/manager/0.log"
Jul 28 19:58:03.563276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.563176    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-b45xm_3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe/manager/0.log"
Jul 28 19:58:03.920081 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:03.920054    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p_90f8dfa8-a03f-4174-865b-33ed4490286e/istio-proxy/0.log"
Jul 28 19:58:04.138669 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:04.138639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-crr5f_5cbaafb5-d683-49f7-ab02-f5c0248eba67/kube-auth-proxy/0.log"
Jul 28 19:58:04.364345 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:04.364311    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/1.log"
Jul 28 19:58:04.474091 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:04.474060    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8sd84_74defc7f-a7f4-4c68-8609-a51b37bd58cf/payload-pre-processing/0.log"
Jul 28 19:58:05.130341 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:05.130309    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vxjmg_9e3d715c-5b16-426a-9fa1-a9b0023bdd5d/main/0.log"
Jul 28 19:58:05.239330 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:05.239304    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ml9hb_728f3d2c-e2c1-4f13-bc33-3759a93e8ea1/main/0.log"
Jul 28 19:58:14.144686 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:14.144653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k5lrq_c78e11fe-9cdb-4d9f-9943-ace27f6ea549/global-pull-secret-syncer/0.log"
Jul 28 19:58:14.328836 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:14.328799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-msf7f_7d16f0ab-d2a4-461b-9291-d24b001c6549/konnectivity-agent/0.log"
Jul 28 19:58:14.401598 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:14.401511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-99.ec2.internal_bb065f00ffb5abb38193a79ef9f99064/haproxy/0.log"
Jul 28 19:58:18.956395 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:18.956363    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/extract/0.log"
Jul 28 19:58:18.982653 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:18.982622    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/util/0.log"
Jul 28 19:58:19.003435 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.003413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zgbh6_6c39614f-c24c-4a96-97fa-1ce73e8122a3/pull/0.log"
Jul 28 19:58:19.036179 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.036156    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/extract/0.log"
Jul 28 19:58:19.056145 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.056120    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/util/0.log"
Jul 28 19:58:19.080803 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.080779    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e052k88_754786e7-24c6-4943-bed5-8ad628c21cef/pull/0.log"
Jul 28 19:58:19.106387 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.106359    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/extract/0.log"
Jul 28 19:58:19.126817 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.126791    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/util/0.log"
Jul 28 19:58:19.151277 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.151254    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4wzr_e0cbc3b7-f1a7-4690-9374-587c47cffdd9/pull/0.log"
Jul 28 19:58:19.183727 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.183706    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/extract/0.log"
Jul 28 19:58:19.206303 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.206281    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/util/0.log"
Jul 28 19:58:19.227224 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.227139    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1x484g_b5da2852-56d0-4ea1-9552-877078061f61/pull/0.log"
Jul 28 19:58:19.778251 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.778224    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h662z_141fde96-b7d5-4b72-8644-5b858660a319/manager/0.log"
Jul 28 19:58:19.803505 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.803485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7v42v_31d2cef2-3c1f-46cf-ba48-b82e67777b4e/manager/0.log"
Jul 28 19:58:19.834855 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.834834    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m9j7c_a86f8dc9-b711-4d33-8a03-193f8fb5d902/kuadrant-console-plugin/0.log"
Jul 28 19:58:19.880278 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:19.880255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xb75x_04b4d565-ae97-480a-98ff-f01f2724b547/registry-server/0.log"
Jul 28 19:58:20.038209 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:20.038122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8k9gc_941ce64b-85c6-48ff-a585-45887464ef72/manager/0.log"
Jul 28 19:58:20.141138 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:20.141106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-b45xm_3b4f8ac9-2cff-4b5c-89fc-9fa6b356eafe/manager/0.log"
Jul 28 19:58:21.872396 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.872359    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/alertmanager/0.log"
Jul 28 19:58:21.893180 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.893158    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/config-reloader/0.log"
Jul 28 19:58:21.915998 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.915968    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/kube-rbac-proxy-web/0.log"
Jul 28 19:58:21.939744 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.939717    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/kube-rbac-proxy/0.log"
Jul 28 19:58:21.963794 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.963768    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/kube-rbac-proxy-metric/0.log"
Jul 28 19:58:21.986452 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:21.986433    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/prom-label-proxy/0.log"
Jul 28 19:58:22.007846 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.007826    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d0f50fa0-d0d7-4db8-bd2c-20e30c7d0650/init-config-reloader/0.log"
Jul 28 19:58:22.226106 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.226019    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p2v_fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a/node-exporter/0.log"
Jul 28 19:58:22.249875 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.249844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p2v_fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a/kube-rbac-proxy/0.log"
Jul 28 19:58:22.273347 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.273328    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p2v_fca36f13-08a2-4ccf-8ca4-7b8bed4ab39a/init-textfile/0.log"
Jul 28 19:58:22.717196 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.717166    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-pgvq7_ae28584c-ec88-4e95-8b40-33df233b25aa/prometheus-operator/0.log"
Jul 28 19:58:22.738720 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.738696    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-pgvq7_ae28584c-ec88-4e95-8b40-33df233b25aa/kube-rbac-proxy/0.log"
Jul 28 19:58:22.764048 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.764022    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-8spsz_61ff5af9-bb62-432a-80e8-9037b97ff515/prometheus-operator-admission-webhook/0.log"
Jul 28 19:58:22.802081 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.802054    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5787969df6-s9p6z_ef112191-715e-4213-8f8f-45f7f27253ce/telemeter-client/0.log"
Jul 28 19:58:22.822369 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.822332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5787969df6-s9p6z_ef112191-715e-4213-8f8f-45f7f27253ce/reload/0.log"
Jul 28 19:58:22.846567 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.846542    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5787969df6-s9p6z_ef112191-715e-4213-8f8f-45f7f27253ce/kube-rbac-proxy/0.log"
Jul 28 19:58:22.876524 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.876499    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/thanos-query/0.log"
Jul 28 19:58:22.896525 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.896505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/kube-rbac-proxy-web/0.log"
Jul 28 19:58:22.918221 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.918200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/kube-rbac-proxy/0.log"
Jul 28 19:58:22.937340 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.937319    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/prom-label-proxy/0.log"
Jul 28 19:58:22.956270 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.956251    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/kube-rbac-proxy-rules/0.log"
Jul 28 19:58:22.975866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:22.975805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b55c6476b-z8vqd_2ffa8b2b-b9c8-445d-be7e-25448a27264a/kube-rbac-proxy-metrics/0.log"
Jul 28 19:58:24.716780 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.716747    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"]
Jul 28 19:58:24.717198 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717184    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717199    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717231    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717251    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e67472-511a-4514-8797-671709717d99" containerName="payload-pre-processing"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717258    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e67472-511a-4514-8797-671709717d99" containerName="payload-pre-processing"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717264    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2ae230-8f6d-4949-a307-f38c52bbd686" containerName="revoke-keys"
Jul 28 19:58:24.717276 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717270    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2ae230-8f6d-4949-a307-f38c52bbd686" containerName="revoke-keys"
Jul 28 19:58:24.717497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717352    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717361    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2ae230-8f6d-4949-a307-f38c52bbd686" containerName="revoke-keys"
Jul 28 19:58:24.717497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717368    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d2a16e-91ae-4418-8099-76a70af96957" containerName="istio-proxy"
Jul 28 19:58:24.717497 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.717374    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e67472-511a-4514-8797-671709717d99" containerName="payload-pre-processing"
Jul 28 19:58:24.722239 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.722202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.726001 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.725976    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cgg7r\"/\"default-dockercfg-jlth6\""
Jul 28 19:58:24.726182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.726015    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"kube-root-ca.crt\""
Jul 28 19:58:24.726182 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.726075    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"openshift-service-ca.crt\""
Jul 28 19:58:24.729423 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.729401    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"]
Jul 28 19:58:24.806092 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.806056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-podres\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.806092 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.806094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-sys\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.806325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.806133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-lib-modules\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.806325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.806160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-proc\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.806325 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.806200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4r57\" (UniqueName: \"kubernetes.io/projected/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-kube-api-access-q4r57\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907509 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4r57\" (UniqueName: \"kubernetes.io/projected/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-kube-api-access-q4r57\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-podres\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-sys\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-lib-modules\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907701 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-proc\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-sys\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-podres\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-lib-modules\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.907871 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.907714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-proc\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:24.915195 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:24.915174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4r57\" (UniqueName: \"kubernetes.io/projected/f7ce17b0-a156-424e-94ed-f137e8a0cc8e-kube-api-access-q4r57\") pod \"perf-node-gather-daemonset-d5lx9\" (UID: \"f7ce17b0-a156-424e-94ed-f137e8a0cc8e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:25.034377 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:25.034292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:25.175943 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:25.175912    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"]
Jul 28 19:58:25.177261 ip-10-0-142-99 kubenswrapper[2577]: W0728 19:58:25.177230    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7ce17b0_a156_424e_94ed_f137e8a0cc8e.slice/crio-9e5d9566ebfddc959d93072d57655abd49b9b278ecc262715d4c001be3b589a4 WatchSource:0}: Error finding container 9e5d9566ebfddc959d93072d57655abd49b9b278ecc262715d4c001be3b589a4: Status 404 returned error can't find the container with id 9e5d9566ebfddc959d93072d57655abd49b9b278ecc262715d4c001be3b589a4
Jul 28 19:58:25.193080 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:25.193049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9" event={"ID":"f7ce17b0-a156-424e-94ed-f137e8a0cc8e","Type":"ContainerStarted","Data":"9e5d9566ebfddc959d93072d57655abd49b9b278ecc262715d4c001be3b589a4"}
Jul 28 19:58:26.200856 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.200815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9" event={"ID":"f7ce17b0-a156-424e-94ed-f137e8a0cc8e","Type":"ContainerStarted","Data":"e0322077955714f8479cf387645a8ceb5e0a1c4b42774d8acd00160e9e326659"}
Jul 28 19:58:26.201303 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.200878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:26.218810 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.218760    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9" podStartSLOduration=2.218745339 podStartE2EDuration="2.218745339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:58:26.21701807 +0000 UTC m=+3373.658938580" watchObservedRunningTime="2026-07-28 19:58:26.218745339 +0000 UTC m=+3373.660665846"
Jul 28 19:58:26.809616 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.809590    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tx22_c2f72dff-7235-4d29-a235-e05baa75c94e/dns/0.log"
Jul 28 19:58:26.828866 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.828841    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tx22_c2f72dff-7235-4d29-a235-e05baa75c94e/kube-rbac-proxy/0.log"
Jul 28 19:58:26.939069 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:26.939042    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2gt4r_34a1b57d-0dca-4762-97af-5cff080bacde/dns-node-resolver/0.log"
Jul 28 19:58:27.548595 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:27.548559    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8m52c_bdaf7e7e-023f-494c-9126-284b229cdf69/node-ca/0.log"
Jul 28 19:58:28.511543 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:28.511506    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p_90f8dfa8-a03f-4174-865b-33ed4490286e/istio-proxy/0.log"
Jul 28 19:58:28.803393 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:28.803312    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-crr5f_5cbaafb5-d683-49f7-ab02-f5c0248eba67/kube-auth-proxy/0.log"
Jul 28 19:58:28.863297 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:28.863248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/0.log"
Jul 28 19:58:28.889337 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:28.889309    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/1.log"
Jul 28 19:58:28.922396 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:28.922367    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8sd84_74defc7f-a7f4-4c68-8609-a51b37bd58cf/payload-pre-processing/0.log"
Jul 28 19:58:29.768445 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:29.768401    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9trb9_560782e8-1ff7-4fa5-b66c-fe5ee1e1f176/serve-healthcheck-canary/0.log"
Jul 28 19:58:30.292972 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:30.292938    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjvj7_a075b9da-2afb-47fa-ac98-f30aac172c3e/kube-rbac-proxy/0.log"
Jul 28 19:58:30.314399 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:30.314368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjvj7_a075b9da-2afb-47fa-ac98-f30aac172c3e/exporter/0.log"
Jul 28 19:58:30.334673 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:30.334636    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jjvj7_a075b9da-2afb-47fa-ac98-f30aac172c3e/extractor/0.log"
Jul 28 19:58:32.216940 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:32.216914    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-d5lx9"
Jul 28 19:58:32.554532 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:32.554455    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8h4zv_cc1aafe3-5f2e-4464-805c-ae1e818c7df8/manager/0.log"
Jul 28 19:58:32.767482 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:32.767447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m9n8w_7e269572-7948-4425-8422-c70da198c1ef/manager/0.log"
Jul 28 19:58:32.922878 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:32.922847    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b7b484f5-fnkst_fac12459-d1ae-4b7f-a88d-5c078997df36/manager/0.log"
Jul 28 19:58:33.021302 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:33.021268    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dh7kc_983016fc-8d6c-4434-a5a7-47107a15cb8f/manager/0.log"
Jul 28 19:58:33.050574 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:33.050551    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-fwshq_1246b0ff-4d7b-42b5-bdf3-f30f5a597482/manager/0.log"
Jul 28 19:58:33.070669 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:33.070645    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-fwshq_1246b0ff-4d7b-42b5-bdf3-f30f5a597482/copy-manifests/0.log"
Jul 28 19:58:34.593560 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.593525    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p_90f8dfa8-a03f-4174-865b-33ed4490286e/istio-proxy/0.log"
Jul 28 19:58:34.656255 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.656209    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8tfg9_85f7f433-db83-40bb-87e5-dbdd784b50ee/openshift-lws-operator/0.log"
Jul 28 19:58:34.871190 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.871159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-crr5f_5cbaafb5-d683-49f7-ab02-f5c0248eba67/kube-auth-proxy/0.log"
Jul 28 19:58:34.908963 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.908932    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/0.log"
Jul 28 19:58:34.934011 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.933982    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/1.log"
Jul 28 19:58:34.955282 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:34.955255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8sd84_74defc7f-a7f4-4c68-8609-a51b37bd58cf/payload-pre-processing/0.log"
Jul 28 19:58:36.387600 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:36.387572    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5bw2p_90f8dfa8-a03f-4174-865b-33ed4490286e/istio-proxy/0.log"
Jul 28 19:58:36.681626 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:36.681538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-crr5f_5cbaafb5-d683-49f7-ab02-f5c0248eba67/kube-auth-proxy/0.log"
Jul 28 19:58:36.716168 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:36.716139    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/0.log"
Jul 28 19:58:36.742768 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:36.742738    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-lrtmn_61d287db-9f2e-4137-9552-18b78a231d2b/istio-proxy/1.log"
Jul 28 19:58:36.763486 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:36.763456    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8sd84_74defc7f-a7f4-4c68-8609-a51b37bd58cf/payload-pre-processing/0.log"
Jul 28 19:58:39.960400 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:39.960359    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh_77642228-1e12-416d-9ddf-a8168eb4823e/kube-storage-version-migrator-operator/1.log"
Jul 28 19:58:39.961479 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:39.961460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-pp2bh_77642228-1e12-416d-9ddf-a8168eb4823e/kube-storage-version-migrator-operator/0.log"
Jul 28 19:58:41.201992 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.201964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/kube-multus-additional-cni-plugins/0.log"
Jul 28 19:58:41.223828 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.223805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/egress-router-binary-copy/0.log"
Jul 28 19:58:41.245609 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.245585    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/cni-plugins/0.log"
Jul 28 19:58:41.284895 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.284870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/bond-cni-plugin/0.log"
Jul 28 19:58:41.315679 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.315651    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/routeoverride-cni/0.log"
Jul 28 19:58:41.338837 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.338811    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/whereabouts-cni-bincopy/0.log"
Jul 28 19:58:41.358791 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.358769    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq6kq_40a25323-a3d4-4cfc-9702-d4c3d1bf5ce7/whereabouts-cni/0.log"
Jul 28 19:58:41.397104 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.397082    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kmc28_e053161e-67d0-4a87-8490-b3b54a7fe458/kube-multus/0.log"
Jul 28 19:58:41.493477 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.493407    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5g9jv_ea2fbe2e-e91e-416c-89d5-2f621a27990b/network-metrics-daemon/0.log"
Jul 28 19:58:41.512873 ip-10-0-142-99 kubenswrapper[2577]: I0728 19:58:41.512853    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5g9jv_ea2fbe2e-e91e-416c-89d5-2f621a27990b/kube-rbac-proxy/0.log"